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Abstract 

The  biennial  Standards  Laboratory  Conference  of  the  National  Conference  of  Standards  Labora- 
tories convened  at  the  Gaithersburg  facilities  of  the  National  Bureau  of  Standards  June  15-17,  1970. 
The  theme  of  the  meeting,  Innovative  Metrology — Key  to  Progress,  was  amplified  by  23  papers 
presented  at  technical  sessions  devoted  to  new  technologies  and  applications,  laboratory  management 
and  operations,  new  methods  of  optimizing  calibration  intervals,  new  ways  of  managing,  and  new 
international  developments. 

Key  words:  Metrology  management;  National  Conference  of  Standards  Laboratories;  physical 
measurement;  Proceedings  NCSL. 
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FOREWORD 


The  growth,  effective  application,  and  wise  regulation  of  technology  depend  upon  the  ability  to 
make  valid  measurements.  Such  measurements  in  turn  depend  upon  the  existence  of  sound  and  evolv- 
ing physical  standards  and  the  effective  functioning  of  a  complex  national  measurement  system,  impor- 
tant elements  of  which  are  the  standards  laboratories  that  comprise  the  National  Conference  of  Stand- 
ards Laboratories. 

The  dramatic  increase  in  the  importance  of  measurement  in  areas  of  great  public  concern  points 
to  future  NBS-NCSL  interactions  on  a  broader  scale  in  keeping  with  the  theme  of  the  1970  Conference 
— innovative  metrology.  In  addition  to  the  traditional  ties  to  NBS  through  involvement  with  the  basic 
measurement  standards,  NCSL's  strengthened  contacts  with  the  measurement  methodology  in  NBS 
areas  of  materials  research  and  applied  technology  will  contribute  materially  to  technological  progress 
that  will  benefit  NBS,  NCSL,  and  the  technological  community  they  serve. 

Because  of  this  mutual  concern  for  measurement,  NBS  is  pleased  to  sponsor  NCSL,  to  host  its 
1970  Conference,  and  to  publish  these  Proceedings.  Primary  responsibility  for  the  technical  content  of 
the  papers  rests,  of  course,  with  the  individual  authors  and  their  organizations. 

Lewis  M.  Branscomb,  Director 
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ON  THE  CONFERENCE  THEME 

J.  L.  Hayes,  NCSL  Chairman 

U.S.  Navy  Metrology  Engineering  Center,  Pomona,  Calif.  91766 


As  noted  in  your  program,  the  theme  of  the  1970 
Standards  Laboratory  Conference  is  "Innovative  Me- 
trology— Key  to  Progress."  For  those  of  you  who  are 
in  a  financial  squeeze  this  year  (and  who  isn't?),  per- 
haps the  theme  would  be  better  stated  as  Innovative 
Metrology — Key  to  Survival!  Either  way  you  put  it, 
the  application  of  innovations  and  fresh  new  ideas  into 
the  management  and  operation  of  the  calibration  and 
metrology  business  is  essential  if  we  are  to  survive  in 
today's  economic  climate.  We  can't  survive  unless  we 
progress  into  more  effective  ways  of  managing  the 
calibration  resources  that  are  at  our  disposal. 

Innovation  is  a  catchword  that  many  of  us  accept 
as  being  full  of  hope  and  excitement;  however,  it  is 
also  full  of  emotional  impact  and  heavily  burdened 
with  responsibility.  To  innovate  is  to  do  more  than 
come  up  with  a  bright  idea.  Innovations  are  impotent 
actions  unless  they  precipitate  valid  change.  Progress 
inevitably  means  change.  Change  begets  fear  of  change; 
resistance  to  change  is  often  the  result.  In  addition, 
natural  inertia  must  be  overcome.  If  a  sufficient  will 
does  not  exist  to  execute  the  innovation,  little  or  no 
gain  results. 

As  managers  of  metrology  operations,  we  all  need 
the  will  to  win,  to  persevere  and  doggedly  to  pursue 
innovations  to  fruitful  conclusions.  We  also  need  the 
courage  to  try,  and  in  trying,  the  courage  to  fail,  if 
need  be,  to  make  significant  contributions.  I  feel  that 
our  present  responsibilities  and  the  theme  of  the  Con- 
ference bring  us  face  to  face  with  a  dual  problem  of 
considerable  magnitude.  We  first  must  have  the  energy 
and  willingness  to  precipitate  innovation.  We  can  ac- 
complish this  by  brainstorming  and  establishing  a 
working  environment  that  is  conducive  to  the  genera- 
tion of  new  ideas  which  are  directed  towards  improv- 
ing our  operations  both  in  quality  as  well  as  in  quan- 
tity. Secondly,  we  then  must  have  even  greater  amounts 
of  determination  to  test  that  environment  by  actually 
bringing  about  the  changes  that  must  be  made  in  spite 
of  the  natural  bureaucracy  of  our  surroundings  and 
the  need  for  security  that  both  our  personnel  and  our- 
selves feel.  This  is  bound  to  take  all  the  wits  we  can 
muster,  the  sensitivity  to  the  effect  this  has  on  the 
people  involved  in  the  change,  and  the  absolute  deter- 
mination to  succeed.  You've  all  seen  it,  so  have  I — 
the  situation  where  someone  has  a  great  idea  but  it 
never  gets  hatched  into  useful  reality  because  no  one 
was  willing  to  suffer  the  real  agonies  of  implementa- 
tion. Perhaps  there's  a  human  simile  that  fits  here:  the 


joys  and  heady  delights  of  innovation  or  conception 
must  be  followed  by  the  pain  and  agony  of  project 
completion  or  childbirth,  the  only  difference  being  that 
on  the  one  hand  we  must  force  it  to  happen;  on  the 
other,  it  occurs  naturally. 

There  is  no  question  in  my  mind  that  we  can  inno- 
vate, nor  is  there  any  question  that  we  must  innovate. 
No  matter  what  the  fiscal  climate  of  our  nation  is,  it 
is  our  responsibility  to  do  our  jobs  in  the  most  cost- 
effective  manner.  That  means  that  we  should  first  chal- 
lenge the  very  jobs  we're  doing  and  confirm  that  they 
are  meaningful.  Once  having  established  that,  we 
then  must  seek  the  most  cost-effective  means  of  execut- 
ing those  jobs  and  of  maintaining  appropriate  quality 
levels  in  doing  so.  Specifically,  operating  cost  reduc- 
tions must  be  achieved  without  a  loss  in  product  qual- 
ity. In  our  case,  we  must  calibrate  instruments  faster 
and  better  and  at  the  same  time  provide  our  customers 
with  test  equipment  which  is  even  more  reliable  during 
its  period  of  use  than  it  is  now.  Is  this  a  wild  idea? 
Maybe.  An  impossible  dream?  No,  sir!  I  contend 
it  can  be  done  and  is  being  done.  It  can  be  done  by 
each  of  you  if  you  have  the  desire.  It  must  be  done 
by  each  of  you  if  you  wish  to  survive  in  your  present 
job  environment  and  if  you  wish  to  make  a  meaning- 
ful contribution  to  your  company  or  organization. 

I  firmly  believe  we  can  together  develop  ways  and 
means  to  achieve  significant  improvements  and  to  save 
our  organizations  significant  financial  resources  if  we 
put  our  minds  and  our  hearts  into  it.  Individually, 
we  may  have  certain  ideas  that  have  not  only  been 
thought  of  but  have  been  tried  and  proven  to  work. 
If  we  can  but  share  these  together,  the  mutual  benefit 
to  all  of  us  within  NCSL  could  be  incredible.  NCSL 
intends  to  initiate  regional  meetings  in  the  fall  of  this 
year  that  will  give  each  of  you  a  chance  to  meet  with 
your  counterparts  from  the  surrounding  area  to  share 
ideas  and  innovations.  Hopefully,  from  each  of  these 
sessions  a  composite  listing  of  ideas,  both  tried  and 
untried,  can  be  collected  and  disseminated  for  possible 
application  within  your  own  organization.  I've  done 
some  of  this  in  my  own  organization  and  the  results 
have  been  exciting  and  full  of  cost-reduction  potential. 
But,  during  the  next  three  days  you'll  hear  of  numer- 
ous innovations  that  you  can  employ.  To  achieve 
progress  through  innovation,  you've  got  to  want  to 
change,  you've  got  to  instill  in  your  people  the  desire 
to  change,  and  you've  got  to  make  the  change.  It's 
up  to  you ! 
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MEASUREMENT  IN  A  CHANGING  WORLD— KEYNOTE  ADDRESS 

Lewis  M.  Branscomb 
Director,  National  Bureau  of  Standards,  Washington,  D.C.  20234 


It  is  a  pleasure  for  me  to  open  this  Conference,  a 
meeting  that  I  believe  will  be  both  fruitful  and  effec- 
tive. 

I  think  it  significant  that  the  group  here  today  is 
not  quite  so  large  as  it  sometimes  has  been.  The  field 
of  metrology,  along  with  every  other  field  of  technical 
endeavor  in  this  country,  seems  to  find  itself  on  the 
defensive  now.  But  I  think  it  would  be  a  mistake 
for  us  to  misconstrue  that  situation:  I  do  not  believe 
that  there  is  any  significant  diminution  in  the  basic 
public  faith  in  science  or  in  the  importance  of  the 
fruits  of  technology  to  the  national  life. 

It  is  true  that  in  the  metrology  field  some  people 
feel  that  the  challenge  of  reliable  quantitative  meas- 
urements may  not  be  quite  as  glamorous  as  space 
travel  or  some  other  dramatic  new  application  of  sci- 
ence. And  sometimes  we  complain  that  measurement 
standards  work  cannot  get  management  attention  or 
that  it  is  too  low  on  the  priority  totem  pole.  Many 
people  end  up  feeling  that  running  a  standards  lab  is 
rather  like  running  a  stock  room,  and  you'll  have  to 
admit  that  there  are  many  standards  labs  that  do  op- 
erate rather  like  a  stock  room — providing  instruments 
that  are  calibrated  with  tender  loving  care  to  people 
who  use  them  in  some  manner  that  may  not  be  well 
known  to  the  man  who  calibrated  the  instrument.  And 
the  standards  lab  that  runs  like  a  stock  room  is  not 
usually  in  a  good  position  to  know  whether  or  not  the 
user  made  effective  use  of  the  effort  that  went  into  the 
original  calibration. 

But  like  everything  else,  the  measurement  standards 
field  is  feeling  the  winds  of  change,  change  that  will 
dramatically  elevate  the  importance  and  the  profes- 
sional standing  of  standards  work.  Public  awareness 
of  measurement  is  also,  I  believe,  going  to  increase. 
I  will  speak  in  a  few  moments  of  the  importance  of 
measurement  in  environmental  and  technological  man- 
agement. But  let  me  parenthetically  call  your  attention 
to  the  fact  that  after  100  years  of  debate  and  11  years 
of  consideration  of  a  specific  measure,  the  Congress 
did,  a  year  and  a  half  ago,  pass  a  Bill  authorizing  a 
study  by  the  Department  of  Commerce  of  the  conse- 
quences to  the  United  States  of  increasing  use  of  met- 
ric measures.  We  have  one  year  left  of  the  three 
given  to  make  the  study,  and  in  August  1971  the  Sec- 
retary of  Commerce  must  report  to  Congress  the  results 
of  our  considerations. 


I  believe  that  this  study,  if  done  properly,  will  shed 
light  on  the  significance  of  measurement  in  the  na- 
tional life.  The  role  of  measurement  is  badly  misun- 
derstood in  most  quarters,  and  only  if  we  understand 
it  properly  can  we  assess  the  consequences  to  the  United 
States  of  changing  our  measurement  language.  That, 
of  course,  is  what  going  to  the  metric  system  really 
means — substitution  of  the  international  system  of 
measurement  for  the  present  mixed  system  of  custom- 
ary units  and  various  versions  of  the  metric  system 
used  in  different  disciplines. 

As  we  examine  the  consequences  of  metrication 
we  will  be  able  to  sort  out  those  parts  of  the  measure- 
ment system  that  concern  the  language  we  use  from 
those  parts  that  actually  influence  the  technology  and 
will  require  changes  in  hardware  in  order  to  adapt  to 
a  change  in  the  measurement  system. 

We  are  carrying  out  this  study  through  different 
methods  for  gathering  information.  One  of  these 
methods  is  a  set  of  national  conferences  opening  in 
August  1970  with  a  meeting  in  Deerfield,  Mass.,  at 
which  we  will  have  heard  the  views  of  the  engineering 
professional  societies  and  several  large  segments  of 
manufacturing  industry.  Other  conferences  will  deal 
with  such  sectors  of  the  economy  as  education,  con- 
sumer problems,  labor,  and  others.  It  is  my  personal 
hope  that  the  metrological  community  will  make  an  ef- 
fort to  provide  us  with  input  on  the  advantages  and 
disadvantages  of  increasing  use  of  metric  measure. 
But  whatever  the  conclusions  of  the  study,  I  believe 
that  as  of  a  year  from  this  summer  we  will  see  major 
attention  being  given  to  the  significance  of  our  meas- 
urement language  in  the  national  life. 

We  are  also  now  seeing  a  dramatic  increase  in  the 
importance  of  effective  standardization,  for  quantita- 
tive, compatible  measurement  is  essential  to  the  wise 
regulation  of  technology  for  the  public  benefit.  The 
last  15  years  have  seen  the  United  States  science  policy 
focus  on  the  exploitation  of  our  scientific  capability 
through  technological  application.  We  have  done  some 
fantastic  things  in  that  period  to  enhance  our  military 
security,  to  increase  our  sources  of  power  production, 
and  perhaps  most  dramatically,  to  make  possible  man's 
first  voyage  to  the  moon. 

But  the  next  10  years  will  be  increasingly  character- 
ized by  the  Government's  effort  to  see  that  science  is 
used  for  the  benefit  of  man  and  that  technology  is 
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mastered  by  mankind  rather  than  becoming  his  master. 
As  a  Nation  we  must  learn  how  to  live  in  harmony 
not  only  with  nature  but  also  with  man's  own  works, 
the  processes  and  the  products  of  his  technology. 

It  is  obvious  that  regulation  of  technology  for  en- 
vironmental improvement  requires  measurements  to 
find  out  what  is  happening  to  the  environment,  to  es- 
tablish technological  alternatives  for  its  management, 
to  determine  the  levels  of  pollution  which  will  be  ac- 
ceptable under  various  circumstances,  to  trace  pollu- 
tion to  its  sources,  and  finally  to  be  the  tool  for  regulat- 
ing environmental  pollution.  For  it  is  clear  that  we 
cannot  eliminate  pollution  of  the  environment.  The 
only  alternative  we  have,  if  we  are  to  expend  energy 
and  not  violate  the  second  law  of  thermodynamics,  is 
to  determine  the  level  and  the  character  of  that  pollu- 
tion in  reurn  for  the  benefits  that  the  polluting  tech- 
nology will  provide  to  us. 

The  challenge  is  to  find  those  kinds  of  pollution, 
whether  they  be  thermal  or  otherwise,  that  are  rela- 
tively acceptable  in  relation  to  the  benefits  to  be  gained. 
This  will  require  an  elaborate  process  of  setting  a 
standard  and  of  regulating  the  performance  of  every 
polluting  organization  or  individual  in  the  society  in 
terms  of  that  standard.  If  the  standard  is  set  on  a 
steep  part  of  the  cost  curve,  then  the  financial  incen- 
tive to  come  close  to  the  edge  of  that  standard  without 
dropping  below  it  will  be  very  great.  And  that  will 
put  great  stress  on  measurements  of  not  only  the  en- 
vironment, but  of  the  technological  process  that  pro- 
duces the  pollution.  Finally,  when  conflicts  over  the 
management  of  the  environment  reach  the  courts,  the 
validity,  the  credibility  of  the  measurement  system  will 
come  under  judicial  review.  And  I  have  said  to  many 
of  my  colleagues  who  have  been  debating  in  recent 
days  about  whether  or  not  there  really  is  an  important 
future  for  measurement  science,  if  you  want  to  make 
yourself  a  bundle,  quit  your  job  and  set  up  a  small 
consulting  company  to  provide  expert  testifiers  to  chal- 
lenge in  court  the  quantitative  creditability  of  our 
measurement  system.  I  believe  that  in  the  environ- 
mental area  the  measurement  capability  leaves  very, 
very  much  to  be  desired,  and  that  under  a  very  severe 
court  challenge  many  of  the  measurements  we  make 
could  be  shown  to  a  jury  to  lack  the  credibility  that  is 
required. 

Concern  over  the  regulation  of  technology  clearly 
goes  far  beyond  environmental  concerns.  Public  at- 
titudes are  fast  changing  concerning  safety  and,  be- 
yond safety,  the  consumer's  pocketbook.  Variations 
in  quality  control  that  were  once  accepted  as  the  nat- 
ural consequence  of  the  vagaries  of  mechanical  systems 
are  no  longer  acceptable.  Society  is  now  asking  that 
we  extract  on  a  routine  production  basis  the  quality  of 
technology  that  we  know  is  in  principle  possible  at  a 
reasonable  price. 

Each  new  area  of  concern  (environmental  pollution, 
consumer  safety  and  protection,  etc.)  will  reflect  itself 
back  through  the  chain  of  engineering  events.  That 
is,  it  will  not  only  concern  the  measurement  of  the  prop- 
erties of  the  product  or  the  amount  of  pollution  in- 
jected into  the  environment,  but  will  go  right  back 


through  production,  production  engineering,  design, 
research,  to  conception  of  the  basic  ideas. 

Thus  the  measurement  system  will  be  called  upon  to 
establish  not  only  credibility,  but  assurance  of  accuracy 
and  control.  Regulatory  teeth  and  major  issues  will 
stand  behind  the  requirement  for  assurance  of  accu- 
racy and  control.  This  will  force  us  to  face  up  to  a 
credibility  gap  in  measurement  standards  work — in 
metrology,  to  use  that  shorthand  label  for  all  of  the 
quantitative  measurement  assurance  services  in  which 
we  are  engaged.  This  gap  is  created  by  a  widespread 
impression  that  the  processes  for  assuring  measurement 
coherence  too  often  obscure  the  purposes,  and  as  a  re- 
sult fail  to  achieve  those  purposes.  So  in  the  interest 
of  conserving  our  resources,  those  of  us  who  are  in 
the  measurement  business  need  to  ask  ourselves  several 
questions:  What  is  the  purpose  of  a  given  measure- 
ment? What  is  the  intended  user's  need?  What  is  his 
measurement  capability?  To  answer  these  questions, 
we  need  a  more  effective  information  feedback  system. 
We  must  get  out  into  the  field  and  find  out  just  what 
is  going  on  so  that  we  can  adapt  what  we  are  supply- 
ing to  what  is  actually  needed  and  used. 

We  at  NBS  must  give  attention  to  several  different 
phases  of  our  responsibilities.  First,  we  are  responsi- 
ble for  establishing  the  central  basis  for  physical  meas- 
urements in  the  United  States — establishing,  improv- 
ing, and  maintaining  these  basic  measurement  stand- 
ards in  our  laboratories.  These  measurements  realize 
the  SI  quantities — the  quantities  defined  in  the  inter- 
national system  of  measurement.  Second,  we  must 
work  down  the  measurement  chain  from  these  funda- 
mental quantities — mass,  length,  time,  temperature, 
luminosity — to  the  hundreds  or  perhaps  even  thou- 
sands of  practical  measurement  processes  that  are 
needed  in  today's  science  and  industry,  such  as  cryo- 
genic flow,  microwave  impedance,  plasma  thermometry, 
and  so  on.  Third,  we  must  work  on  the  problem  of 
transferring  into  the  secondary  laboratories  through- 
out the  United  States  the  basic  measurement  capability 
that  we  generate  here.  Fourth,  we  must  assure  ade- 
quate verification  that  the  transfer  is  sufficiently  effec- 
tive for  the  user's  requirement.  This  requires  an  ac- 
tive, outward-looking  role  on  our  part. 

There  is  no  reason  for  NBS  not  being  able  to  trans- 
fer its  accuracy  under  all  the  conditions  required  into 
the  laboratories  of  users  in  such  a  way  that  the  user 
knows  that  he  is  tied  back  to  the  Bureau  in  an  oper- 
ationally effective  way.  Every  other  standards  labora- 
tory has  a  precisely  parallel  responsibility  with  re- 
spect to  its  clients,  and  to  that  extent  all  of  you  share 
with  us  the  same  nature  of  responsibility.  Let  me  give 
you  a  couple  of  recent  examples  to  indicate  some  of 
the  things  we  are  doing  in  two  of  those  four  responsi- 
bilities I  named — those  having  to  do  with  the  estab- 
lishment of  standards  and  their  more  effective  dissem- 
ination. 

In  length  measurement,  technology,  through  develop- 
ment of  the  laser,  has  again  come  to  revolutionize 
measurement  science  as  measurement  science  in  turn 
pulls  and  leads  technology.  The  laser  obviously  has 
not  only  given  the  promise  but  the  actuality  of  revo- 
lutionizing precision  distance  measurement  because  of 
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its  particularly  attractive  properties  of  coherent  light 
of  very  high  brightness.  We  have  recently  in  our  re- 
search program  found  a  way  to  stabilize  a  laser  to  an 
accuracy  of  about  one  part  in  10".  We  have  in  fact 
built  two  completely  separate  lasers  each  oscillating  in 
the  near  infrared  at  a  frequency  of  about  1014  Hz,  in- 
dependently stabilized  them  with  completely  separate 
apparatuses,  beat  them  together,  and  found  the  beat 
frequency  to  be  only  about  100  Hz.  This  development 
can  be  carried  a  good  bit  further.  We  have  the  pros- 
pect of  a  wavelength  standard  whose  reproducibility 
is  nearly  competitive  with  the  reproducibility  of  the 
frequency  standard  itself.  Close  on  the  heels  of  that 
development  is  the  real  likelihood  that  in  the  next  year 
or  two  our  scientists  will  succeed  in  multiplying  the 
frequency  standard  all  the  way  up  to  the  optical  range, 
permitting  direct  comparison  of  the  standard  of  wave- 
length with  the  standard  of  time  interval.  At  that 
point  we  will  have  the  opportunity,  if  we  wish,  to  assign 
an  arbitrary  value  to  the  speed  of  propagation  of  elec- 
tromagnetic radiation,  and  discard  either  the  standard 
length  or  the  standard  of  time  at  our  convenience. 
This  is  an  example  of  how  rapidly  the  scientific  basis 
for  measurement  is  proceeding. 

Let  me  now  mention  a  new  contribution  that  we  are 
making  in  time  standards  dissemination.  Many  people 
are  under  the  assumption  that  the  two  most  essential 
uses  of  time  are  the  measurement  of  time  interval  or 
elapsed  time,  and  the  determination  of  absolute  time 
in  relation  to  the  motions  of  the  earth  in  its  astronomi- 
cal context.  The  former  obviously  is  essential  for  any 
physical  measurement  in  which  time  enters  as  a  param- 
eter because,  except  in  the  field  of  cosmology,  time 
enters  physical  processes  as  a  time  interval.  Absolute 
time,  or  epoch,  is  vital  to  navigation,  since  we  navigate 
on  the  surface  of  the  globe  with  respect  to  geographic 
coordinates.  It  is  obviously  convenient  to  do  so  in 
relation  to  the  position  of  the  surface  of  the  earth  in 
its  external  environment  of  the  moon,  planets,  and 
stars.  But  it  is  increasingly  important  in  technology 
to  be  able  to  achieve  simultaneity,  which  means  the 
absence  of  a  time  interval  between  two  remote  loca- 
tions. Simultaneity  is  now  of  very  great  importance 
in  many  applications.  Included  among  them  are  prob- 
lems of  space  navigation,  of  collision  avoidance  for 
aircraft,  and  in  telecommunications.  Pulse  code  mod- 
ulation communications  are  made  possible  if  there  is 
an  independent  time  base  at  all  points  in  the  system 
so  that  the  train  of  pulses  can  be  decoded  at  the  re- 
ceiving end. 

Two  of  the  ways  of  achieving  simultaneity  around 
the  country  are  to  proliferate  atomic  clocks  or  to  es- 
tablish a  satellite  time  signal  system.  You  can  pro- 
liferate atomic  clocks,  since  every  man's  cesium  is  the 
same  as  the  next  man's,  but  the  capital  investment  in 
clocks  is  rather  prohibitive.  The  clocks  must  be  car- 
ried from  place  to  place  to  keep  them  synchronized. 
Carrying  clocks  from  one  place  to  another  is  of  course 
used  to  establish  a  coordinated  time  base  for  independ- 
ent laboratory  clocks,  but  this  process  is  also  expen- 
sive. The  satellite  system  is  a  perfectly  practical  means 
of  establishing  simultaneity  technologically,  but  it  is 
even  more  expensive. 


Our  laboratories  in  Boulder  are  now  experimentally 
investigating  the  feasibility  of  another  way  of  estab- 
lishing simultaneity  through  color  television.  The  idea 
is  basically  a  very  simple  one.  The  three  major  net- 
works in  New  York,  in  transmitting  their  color  pic- 
tures, generate  a  very  accurate  ZV2  -megahertz  subcar- 
rier.  It  is  possible  to  put  a  modulation  on  the  sub- 
carrier  during  the  interval  of  time  between  pictures. 
There  are  several  lines  on  the  TV  screen  on  which  one 
can  impress  a  code  that  will  transmit  a  time  signal 
throughout  the  television  network.  An  encoder  at- 
tached to  the  TV  set  displays  numeric  time  information 
on  the  screen.  It  is  then  only  necessary  for  a  given 
laboratory  to  determine  once  what  the  phase  delay  is 
between  its  location  and  the  source  of  this  coded  sig- 
nal, in  order  for  the  television  signal  to  establish  the 
time  difference  between  the  absolute  time  reference 
maintained  by  NBS  and  the  laboratory's  local  clock. 
This  provides  a  running  correction  for  the  local  clock 
as  well  as  a  basic  time  base  in  hours,  minutes,  sec- 
onds. This  system  is  being  evaluated  experimentally 
over  TV  stations  in  Denver  and  Washington.  It  is  also 
being  used  to  provide  the  frequency  comparison  be- 
tween the  Naval  Observatory  clocks  in  Washington  and 
those  of  our  laboratories  in  Boulder.  We  have  per- 
mission from  the  FCC,  AT&T,  and  the  major  TV  net- 
works to  proceed  with  the  examination  of  this  system 
with  a  view  to  implementation  on  a  national  basis.  We 
hope  to  evolve  a  dissemination  system  which  can  pro- 
vide synchronization  accuracy  of  a  microsecond  or 
better  in  any  laboratory  which  otherwise  would  have 
to  accept  the  millisecond  precision  inherent  in  real- 
time use  of  the  ionospherically  propagated  signal  from 
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Beyond  improving  the  basic  standards  and  means 
for  transferring  those  standards  effectively  into  the 
laboratory  of  the  user,  we  must  study  the  whole  proc- 
ess of  measurement  in  order  to  ensure  that  it  serves 
its  intended  purpose.  Some  people  think  of  a  meas- 
urement in  terms  of  a  measuring  instrument.  I  think 
the  word  "calibration"  tends  to  convey  that  image. 
But  a  measurement  can  be  understood  only  in  terms 
of  a  total  man-device  system.  Heisenberg,  in  develop- 
ing the  uncertainty  principle,  taught  us  that  there  is 
really  no  meaningful  measurement  that  does  not  affect 
the  system  being  measured.  We  must  also  recognize 
that  the  human  beings  involved  in  the  measurement 
system  are  an  important  influence  on  the  measurement 
as  well  as  being  influenced  by  it.  We  must  also  in- 
corporate the  people  making  and  interpreting  a  meas- 
urement in  our  understanding  of  a  measurement. 

A  measurement  results  when  we  have  a  theory  for 
the  measurement,  and  know  the  system  response  in  re- 
lation to  a  range  of  system  circumstances — all  those 
that  might  show  up  in  the  measurement,  including  a 
subset  of  special  circumstances  called  calibration.  This 
special  subset  of  circumstances  gives  the  restrictions 
needed  when  the  system  is  configured  to  permit  a 
theoretical  tie  back  to  the  basic  measurement  stand- 
ards. In  the  determination  of  physical  properties  car- 
ried out  in  many  laboratories  all  over  the  country,  this 
philosophy  is  too  often  ignored  and  as  a  result  there 
is  entirely  inadequate  attention  given  to  systematic  er- 
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rors  involved  in  absolute  measurements  of  all  kinds; 
errors  that  derive  not  only  from  the  intrinsic  proper- 
ties of  the  device  being  used  but  also  from  the  man- 
ner and  skill  with  which  it  is  used.  It  is  important  to 
determine  when  absolute  accuracy  is  really  needed. 
Then  we  must  ensure  that  we  really  get  what  we  re- 
quire. In  the  past  we  have  too  often  accepted  a  cali- 
bration system  that  failed  to  assure  this  accuracy  in 
actual  practice.  Perhaps  such  failures  have  occurred 
in  part  because  of  realization  that  requested  accura- 
cies were  not  always  required. 

In  the  area  of  materials  management  we  have  an  at- 
tractive method  of  transferring  accuracy  from  a  central 
standards  laboratory  into  the  field.  This  is  the  de- 
termination of  certain  properties  of  matter  in  absolute 
units  with  known  precision  and  accuracy.  We  are  es- 
tablishing here  at  the  Bureau  a  program  on  such 
"bench  mark  data"  devoted  to  a  properly  documented 
determination  of  selected  properties.  And  we  will  not 
use  the  traditional  standards  of  excellence  of  the  scien- 
tific literature  for  judging  the  quality  of  our  work  in 
this  area.  Instead  we  treat  a  bench  mark  data  prob- 
lem just  as  any  systems  engineering  problem,  in  which 
you  set  up  the  necessary  procedures  for  documenting 
that  such  accuracy  has  been  achieved  well  enough  to 
satisfy  any  group  of  skeptics.  I  mention  this  program 
because  the  scientific  community  accepts  the  idea  that 
work  may  be  done  and  published  as  correct  and  yet 
still  not  be  believed  by  the  peers  of  the  scientist  per- 
forming the  work.  In  the  laboratories  of  those  con- 
cerned with  the  measurement  system  we  cannot  ac- 
cept a  system  in  which  measurements  are  not  accepted 
as  credible.  In  the  establishment  of  absolute  data,  it 
is  essential  that  we  learn  how  to  document  an  experi- 
ment so  that  it  can  stand  on  its  own. 

A  consistent  national  measurement  system  properly 
coordinated  with  the  system  of  other  nations  depends 
on  several  things:  the  existence  of  a  theory  of  measure- 
ment based  on  the  relationships  among  physical  quan- 
tities; the  establishment  of  a  set  of  units  and  reference 
standards;  the  development  of  a  set  of  technical  pro- 
cedures for  comparing  a  quantity  to  be  measured  with 
the  measurement  unit;  and  finally,  the  development  of 
measurement  equipment  for  carrying  out  such  com- 
parisons and  for  determining  multiples  and  submulti- 
ples  of  basic  units.  Too  often  in  the  past  the  thinking 
of  metrologists  has  been  dominated  by  such  matters 
as  traceability,  calibration  certificates,  accuracy  charts, 
fee  schedules,  echelons,  calibration  intervals,  accuracy 
ratios,  and  class  tolerances.  These  are  material  re- 
quirements or,  if  you  like,  design  requirements,  as  op- 
posed to  the  performance  requirements  that  we  would 
like  to  apply  to  the  measurement  system. 

To  illustrate  the  inadequacy  of  merely  requiring 
traceability,  consider  the  following  situation  that  could 
occur  and  indeed  does  occur  in  the  area  of  mass  meas- 
urements. Suppose  a  laboratory  has  its  reference 
standards  regularly  calibrated  by  NBS,  by  shipping 
them  here  and  having  them  measured  here.  And  hav- 
ing the  best  conditions  for  storing  its  weights,  so  they 
are  properly  cared  for,  the  laboratory  finds  that  there 
is  essentially  no  change  in  the  values  as  measured  by 
the  NBS  over  an  extended  period  of  time.    Surely  this 


is  traceability  at  its  best.  Yet  the  measurements  made 
by  such  a  laboratory  may  be  subject  to  great  error  if 
environmental  controls  in  the  user  laboratory  are  not 
adequate,  or  if  procedures  are  not  properly  followed, 
or  the  measurement  equipment  is  in  some  way  faulty. 
Stated  another  way,  we  may  know  all  about  the  lab- 
oratory's standards,  but  still  know  little  about  its  abil- 
ity to  measure. 

I  was  even  told  a  few  days  ago  of  a  standards  lab- 
oratory which  was  making  repeated  traceability  certifi- 
cation of  the  standards  of  a  subordinate  laboratory. 
This  subordinate  laboratory  seemed  to  have  an  extra- 
ordinarily good  record,  as  its  standards  showed  little 
deviation  from  year  to  year.  Then  it  was  discovered 
that  the  subordinate  laboratory  had  not  even  unpacked 
the  standards  it  submitted  to  the  central  laboratory! 
Its  staff  just  left  them  in  the  box  and  shipped  them 
back  for  recalibration  the  next  year.  Well,  that  is  per- 
fection in  traceability  and  zero  in  measurement  assur- 
ance! 

Concern  over  calibration  interval,  accuracy  ratio, 
and  traceability  is  quite  popular  in  the  maintenance 
of  standards,  and  standards  are  a  vital  element  in  a 
consistent  system  of  measurement.  But  standards  alone 
are  not  enough  to  meet  the  demands  of  modern  tech- 
nology. 

The  interchangeability  requirements  of  industrial 
processes,  the  compatibility  and  authenticity  require- 
ments of  guidance  and  control  systems,  and  the  com- 
patibility of  measurement  controls  in  commerce  are 
all  examples  of  a  need  for  a  coordinated  measurement 
system.  However  it  is  the  kind  of  measurements  made 
by  the  system  and  not  just  the  quality  of  the  standards 
that  is  at  issue.  Measurements  in  a  physically  con- 
sistent system  for  which  the  requirements  of  compati- 
bility and  authenticity  apply  are  incompletely  and 
sometimes  even  incorrectly  characterized  by  the  prop- 
erties of  the  standards  used.  Our  focus  then  should 
be  on  the  measurements  that  count  and  the  process 
that  generates  those  measurements. 

Although  NBS  establishes  measurement  standards 
for  some  50  basic  physical  quantities,  the  transfer  from 
these  standards  to  practical  measurements  is  a  respon- 
sibility resting  primarily  with  users.  However,  the 
Bureau  must  know  the  user's  measurement  needs,  so 
that  it  can  be  responsive  to  providing  standards  and 
methodology  that  are  suited  to  those  needs.  The 
transfer  of  measurement  capability  can  be  achieved 
with  the  usual  calibration  approach,  but  for  the  high- 
est accuracy  work  the  laboratory  may  need  to  set  up 
an  independent  realization  of  the  basic  units  from 
their  fundamental  definitions.  Or  more  economically, 
a  laboratory  may  tie  to  the  Bureau  by  comparative 
procedures  so  that  that  independent  laboratory  can  be 
as  good  as  if  the  Bureau  had  set  up  an  alternative 
laboratory  of  its  own  on  their  site. 

It  is  the  answer  to  the  following  question  that  we 
seek  as  a  new  approach  for  the  dissemination  of  meas- 
urement capability:  What  procedures,  how  much  work, 
what  kind  of  control,  what  performance  criteria  would 
be  needed  to  constitute  evidence  as  to  the  quality  of 
measurements  of  a  laboratory?  Practical  measure- 
ment can  be  easily  described  in  terms  of  its  elements, 
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the  concept  of  the  quantity  to  be  measured,  the  perti- 
nent physical  laws,  the  various  instruments  used,  the 
physical  standards,  the  operators  and  the  appropriate 
procedures,  the  environment  in  which  the  measure- 
ment is  to  be  carried  out,  the  computations  to  be  made, 
and  finally  the  means  of  establishing  parameters  of 
performance. 

The  measurement  process  is  essentially  a  production 
process,  the  product  being  numbers — that  is,  measured 
values.  A  well-known  characteristic  of  a  measurement 
process  is  that  repeated  measurements  of  the  same 
thing  result  in  non-identical  numbers.  To  specify  a 
measurement  process  involves  ascertaining  the  limit- 
ing mean  of  the  process,  its  variability  or  imprecision 
due  to  imperfections  in  the  system,  the  possible  extent 
of  systematic  errors  or  bias  from  identifiable  sources, 
and  the  overall  limits  to  the  uncertainty  of  independent 
measurements. 

When  we  have  tolerances  of  ±  5  microinches  on 
ball-bearing  diameters,  the  uncertainty  in  our  meas- 
urements may  be  a  large  part  of  the  tolerance.  We 
have  to  cope  with  the  problem  of  stating  in  numbers 
the  degree  to  which  we  have  achieved  the  desired  goal. 
In  all  cases,  the  metrologist  must  understand  the  real 
goal  of  measurement  effort.  For  example,  in  the  case 
of  the  ball  bearing,  the  important  element  may  be  the 
need  for  a  matched  set  of  balls  of  identical  diameter, 
and  it  may  be  possible  to  produce  sets  of  balls  that 
match  to  much  better  tolerances  than  one  would  assign 
to  the  value  of  an  individual  ball  in  an  interlaboratory 
comparison  study. 

Another  example  is  in  the  measurement  of  light  out- 
put from  fluorescent  tubes — the  product  of  a  1  billion 
dollar  industry.  Here  it  appears  that  the  marketplace 
hinges  on  1  percent  differences  in  levels  of  performance 
claimed  by  manufacturers — that's  30  lumens  out  of 
3,000.  On  the  other  hand,  we  do  not  have  sufficient 
confidence  in  our  basic  standards  here  at  the  Bureau 
to  assure  us  that  it  is  possible  to  measure  the  light 
emitted  by  a  fluorescent  tube  to  sufficient  accuracy  to 
ensure  a  one  percent  transfer  of  accuracy  into  the 
industry. 

And  it  may  be  that  in  this  industry  not  only  the 
accuracy  but  even  the  precision  with  which  fluorescent 
tube  output  can  be  measured  is  not  as  good  as  1  per- 
cent. But  the  question  that  we  must  ask  is  "Is  it  nec- 
essary to  try  to  have  every  lamp  manufacturer  in  a 
position  to  measure  fluorescent  tube  output  in  absolute 
units — that  is,  in  lumens  to  better  than  1  percent?  Or 
is  it  sufficient  to  ensure  harmony  in  the  industry  with 
respect  to  luminosity  precision,  allowing  normal  mar- 
ket forces  to  determine  the  adequacy  of  the  level  of 
light  output?"  These  questions  are  not  easy  to  an- 
swer because  they  hinge  not  only  on  direct  cost  studies, 
but  on  questions  of  confidence  in  the  market  process 
to  which  measurement  has  an  important  contribution. 

The  key  element  in  evaluating  cases  of  this  kind  is 
that  of  obtaining  a  sequence  of  measurements  of  the 
same  quantity.  In  standards  laboratories  this  is  fairly 
easy  to  achieve  simply  by  remeasuring  one  of  the  lab- 
oratory standards  as  if  it  were  an  unknown,  or  using 
one  or  more  items  typical  of  the  test  items  for  the  same 
purpose.    The  routine  calibration  of  the  same  object 


used  as  a  check  standard  tells  us  what  the  measurement 
process  can  do.  It  is  not  just  a  simulation  of  the  cali- 
bration process — it  is  the  real  thing,  without  the  need 
for  any  assumptions.  Continued  measurement  of  the 
check  standard  provides  the  basis  for  the  precision 
statement  and  correlating  its  behavior  with  environ- 
mental procedures  or  other  factors  enables  us  to  meas- 
ure or  set  bounds  for  possible  systematic  error.  For 
control  of  the  measurement  process,  this  check  stand- 
ard need  not  have  the  stability  one  would  require  of 
members  of  the  standard  group.  Predictability  of  its 
performance  is  the  basic  requirement. 

Having  established  control  in  one's  own  laboratory 
one  then  asks — what  is  the  most  effective  way  of  co- 
ordinating my  measurement  process  with  others  in  the 
national  measurement  system?  The  problem  is  that 
of  transfer  of  measurement  capability  and  again  one 
has  to  study  this  transfer  mechanism  as  a  process  and 
determine  its  operating  characteristics  just  as  was  done 
for  measurements  in  his  own  laboratory. 

The  Bureau  is  considering  new  approaches  for  the 
transfer  of  measurement  capability  in  selected  areas  to 
avoid  the  wasteful  duplication  implied  by  having  each 
user  develop  his  own  procedures  for  assuring  the  ade- 
quacy of  the  transfer.  For  example,  in  voltage  stand- 
ards the  Bureau  is  engaged  in  a  program  of  develop- 
ing procedures  using  shippable  standards  for  trans- 
ferring the  value  from  the  national  reference  group  to 
other  primary  laboratories.  The  old  practice  of  mov- 
ing the  customer's  reference  group  to  NBS  for  cali- 
bration left  unanswered  the  question  of  whether  the 
group  survived  the  return  trip  unchanged,  and  placed 
on  the  user  the  burden  for  the  study  of  the  transfer 
mechanism. 

The  first  or  pilot  program  of  this  type  was  in  the 
calibration  of  mass  where  the  question  of  transporta- 
bility of  the  standard  does  not  arise.  We  first  estab- 
lished that  a  state  of  statistical  control  existed  in  our 
own  laboratory.  Participating  laboratories  were  re- 
quired to  evaluate  their  own  performance  parameters, 
and  with  the  aid  of  a  pair  of  kilogram  standards  bor- 
rowed from  the  Bureau  could  know  to  what  degree 
their  work  would  be  compatible  with  others  in  the 
system.  A  fundamental  difference  from  their  usual 
procedures  is  the  emphasis  on  the  continued  verifica- 
tion of  control  within  the  laboratory  and  between  it 
and  other  parts  of  the  measurement  system.  This  tells 
the  laboratory  how  well  it  performs — and  is  more 
meaningful  than  a  certificate  that  says  how  well  some 
other  standards  laboratories  performed.  The  amount 
of  effort  is  dictated  by  the  measurement  needs  of  the 
user  and  not  by  tradition.  Similar  efforts,  which  we 
call  Measurement  Analysis  Programs,  are  under  way 
for  voltage,  resistance,  capacitance,  temperature,  length, 
and  several  other  quantities. 

To  optimize  our  efforts  in  serving  the  national  meas- 
urement system,  we  must  look  at  such  objectives  of  our 
calibrations  as  achieving  compatibility  among  a  num- 
ber of  laboratories.  Even  if  an  individual  in  a  lab- 
oratory sends  a  device  to  the  Bureau,  and  we  make 
the  assignment  of  its  value  with  the  smallest  possible 
error,  his  job  is  not  done.  For  example,  if  he  sets  up 
his  balance  in  a  breezy  hallway,  his  measurements  are 
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not  necessarily  improved  by  having  a  super-accurate 
NBS  mass  calibration.  He  still  has  to  evaluate,  in  his 
own  laboratory,  the  component  of  error  due  to  his 
breezy  hallway.  In  cases  where  the  device  may  be  al- 
tered by  the  trip  to  the  standards  laboratory  his  knowl- 
edge will  be  no  better  than  the  evidence  in  his  labora- 
tory that  it  has  survived  the  trip. 

If  there  are  problems  in  comparative  measurements 
or  in  transporting  the  standards  it  may  be  better  that 
we  develop  standards  that  we  can  ship  to  the  other 
laboratories.  Then  they  can  calibrate  their  reference 
standards  on  their  own  site  by  using  their  own  com- 
parator and  procedures,  and  all  the  associated  sys- 
tematic errors  will  affect  this  transfer  to  the  same 
extent  as  it  will  their  future  measurements.  This 
eliminates  a  source  of  bias  that  would  be  present  if 
the  reference  standards  were  calibrated  at  the  standards 
laboratory.  The  user  then  works  only  as  hard  as  he 
needs  to  in  order  that  the  error  in  the  local  inter- 
comparison  be  as  small  as  required,  while  NBS  con- 
centrates on  the  maintenance  of  the  absolute  units  and 
the  problem  of  the  transport  of  those  units  to  his 
laboratory. 

These  examples  are  illustrative  of  our  desire  to  make 
sure  that  our  measurements  are  not  only  accurate  but 
also  optimally  relevant  to  the  actual  end  use  of  the 
measurements.  The  measurement  process  is  a  highly 
complicated  one  and  needs  to  be  evaluated  constantly 


as  to  efficiency  and  effectiveness,  so  that  the  services 
that  you  and  we  provide  will  meet  neither  more  nor 
less  than  is  required  to  satisfy  the  needs  of  users. 

My  remarks  have  ranged  from  generalizations  to 
technical  details  of  the  measurement  process.  I  wanted 
to  emphasize  the  fact  that  any  standards  laboratory 
has,  I  believe,  the  same  responsibility  that  we  now 
are  increasingly  feeling — namely,  the  assurance  of  the 
effective  transfer  of  measurement  into  the  hands  of 
users.  And  I  believe  that  this  requires  a  relationship 
between  the  standards  laboratory  and  the  user  in  which 
the  people  and  the  system  are  coupled  and  the  stand- 
ards laboratory  has  the  opportunity  and  the  obligation 
to  evaluate  the  effectiveness  of  the  user's  measure- 
ment. Once  a  standards  laboratory  is  in  a  position  to 
do  that,  then  that  laboratory  becomes  an  effective  and 
indeed  an  indispensable  tool  of  management.  We  in 
the  Bureau  are  finding  in  various  areas  of  technology 
that  the  integrity  of  the  measurement  system  is  increas- 
ingly impacting  the  policy-making  levels  of  govern- 
ment concerned  with  the  regulation  of  technology.  If 
a  standards  laboratory  is  in  a  position  not  only  to 
calibrate  instruments,  but  to  assess  the  quality  of  the 
job  being  done  by  those  who  are  making  measure- 
ments of  importance  to  the  laboratory's  sponsor  or 
clients,  then  that  standards  laboratory  becomes  an 
important  element  of  overall  management  and  tech- 
nological control  in  the  client  institution. 
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This  appearance  marks  a  new  experience  for  me. 
When  I  talk  to  engineering  or  educational  groups  I  can 
always  say,  "I  feel  right  at  home  here  because  I  have 
been  in  this  business."  Tonight  for  the  first  time  I 
can  say  to  a  group,  "I  feel  right  at  home  here  because 
I  run  a  standards  laboratory  too."  The  word  "run" 
is  a  bit  strong  for  in  a  real  sense  no  one  as  far  from 
the  bench  as  I  am  can  "run"  a  precision  laboratory. 
But  my  pride  in  this  association  encourages 
exaggeration. 

I  am  tempted  to  say,  "My  standards  laboratory  is 
better  than  your  standards  laboratory,"  but  that  would 
not  be  fair.  It  is,  but  by  definition,  not  necessarily  by 
any  particular  effort  of  mine. 

I  have  often  said  that  what  you  purchase  from  a 
standards  laboratory  is  negentropy — the  reduction  of 
uncertainty.  We  tend  to  think  of  that  quantity  in 
relation  to  the  standards  laboratory  in  a  rather  re- 
stricted sense.  But  it  has  an  extended  sense,  and  that 
sense  of  negentropy  has  a  good  deal  to  do  with  the 
reasons  for  which  we  are  gathered  here  this  week. 

It  is  accepted,  certainly  by  this  group  at  any  rate, 
that  civilization  is  built  on  measurement.  Civilization 
is  also  built  on  negentropy.  Our  ability  to  plan  and  to 
build  depends  on  our  ability  to  know  the  limitations 
of  things,  of  devices,  of  institutions,  and  of  processes. 
So  in  this  sense  the  standards  laboratory  does  indeed, 
within  its  scope,  provide  the  very  stuff  of  civilization. 

As  a  main  prop  of  civilization,  the  standards  labora- 
tory is  subject  to  the  forces  and  currents  of  that 
civilization.  One  of  the  main  currents  of  our  civiliza- 
tion, and  one  of  its  tribal  gods,  is  innovation.  And 
so  we  are  gathered  here  to  perform  a  ritual  soul- 
searching  in  quest  of  that  elusive,  magical  power 
which  resides  in  all  of  us  and  merely  needs  to  be 
bidden  to  spring  forth  and  grant  its  bounties. 

Unfortunately,  my  experience  as  engineer,  educator, 
and  administrator  has  shown  me  that  innovation,  like 
most  gods,  is  a  capricious  servant.  Once  summoned, 
it  has  a  way  of  overpowering  sense  and  reason,  of 
demanding  an  allegiance  beyond  that  which  benefits 
the  believer.    It  becomes  an  end  in  itself. 

This  aspect  of  the  innovation  problem  is  balanced 
by  the  fact  that  few  people,  even  those  who  talk  and 
promote  innovation,  really  want  it.  Innovation  dis- 
rupts our  routines  and  our  security.    It  challenges 


our  competence,  and  there  are  few  of  us  who  will  put 
up  with  that  sort  of  thing. 

So  there  are  two  conflicting  currents  here  which  the 
management  of  a  standards  laboratory,  like  any  other 
modern  enterprise,  must  keep  in  balance — the  impulse 
to  change,  and  the  resistance  to  change.  The  key  to 
that  balance  is  being  able  to  recognize  what  constitutes 
progress — not  change  or  innovation,  but  progress. 
Change  for  change's  sake  must  be  avoided;  but  no 
change  which  promises  improved  operations  and 
services  should  be  lost. 

These  considerations  are  common  to  all  management, 
not  just  to  the  standards  business.  But  in  the  stand- 
ards and  calibration  activity  we  have  an  added  dimen- 
sion which  makes  the  decision-making  process  more 
difficult  and  makes  desirable  changes  perhaps  more 
difficult  to  introduce.  That  added  dimension  is  the 
great  degree  of  routine  and  established  procedure 
which  a  standards  laboratory  must  have  to  insure  the 
accuracy  and  repeatability  of  its  results.  Obviously 
these  routines  must  not  be  disturbed  for  light  and 
transient  causes. 

Neither  should  the  people  in  a  standards  operation 
be  disturbed  unnecessarily.  They  are  a  special  breed 
with  an  orientation  toward  and  a  commitment  to  ex- 
cellence. They  will,  rightly,  resist  any  incursion  into 
the  quality  of  their  work. 

How  does  the  manager  of  a  standards  and  calibra- 
tion operation  keep  it  moving  forward  with  all  de- 
liberate speed  in  full  recognition  of  these  realities?  It 
seems  to  me  the  first  necessity  is  a  firm  understanding 
of  his  operation  and  what  it  can  and  should  do. 

What  do  we  expect  of  a  first-rate  standards  labora- 
tory? Two  things  stand  out  in  my  mind.  First  is  the 
ability  to  give  the  customer  an  unambiguous  statement 
of  what  the  laboratory  has  done  for  him.  Second  is 
the  provision  of  a  range  of  services  to  meet  the  varying 
needs  of  various  customers.  Some  customers  want 
the  utmost  in  accuracy  in  a  calibration;  some  are 
willing  and  able  to  sacrifice  a  decimal  place  or  two  for 
a  quicker  turnaround  time;  many  are  not  so  interested 
in  the  extremes  of  accuracy  as  they  are  in  extreme 
certainty  of  the  limits  of  inaccuracy.  A  first-rate 
standards  laboratory  must  be  able  to  meet  the  needs  of 
this  range  of  customers  and  to  be  able  to  tell  them 
exactly  what  they  are  getting  for  their  money. 
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It  is  up  to  the  manager  to  make  sure  that  his  labo- 
ratory can  meet  the  demands  made  on  it  in  a  sound 
and  economical  framework — that  is,  in  terms  of  the 
usual  management  decisions  which  must  balance 
services  against  investment  in  floor  space,  hardware, 
personnel,  and  so  on. 

Even  as  the  manager  develops  for  himself  a  good 
idea  of  what  his  organization  is  about,  he  must  com- 
municate that  idea  to  the  people  who  work  for  him. 
If  he  has  an  obligation  to  provide  his  customers  with 
an  unambiguous  statement  of  what  the  laboratory  is 
about,  he  owes  the  same  to  his  people.  The  more 
clearly  the  aims  of  the  laboratory  are  stated  and  under- 
stood, the  more  surely  the  individual  employee  can 
relate  himself  to  the  mission  and  feel  a  part  of  it. 

What  I  am  advocating  here  is  that  a  laboratory 
should  have  a  clear  set  of  performance  criteria  for 
itself  and  its  employees.  This  is  a  necessity  for  know- 
ing where  you  are  and  where  you  intend  to  go.  More 
to  the  point,  these  performance  criteria  give  you  a 
basis  for  innovating.  Like  all  performance  criteria, 
they  make  it  possible  to  evaluate  new  technology  in 
objective  terms  to  see  if  it  advances  the  state  of  the  art 
or  is  merely  change  for  the  sake  of  change.  Per- 
formance criteria  enable  the  laboratory  to  make  de- 
sirable innovations  without  undue  disruption  in 
operations  and  service. 

I  think  you  can  understand  my  insistence  that  you 
"know  thyself."  It  is  only  to  the  extent  that  you  under- 
stand your  own  operation  that  you  can  manage  for 
innovation.  But,  how  many  of  you  really  know  the 
trade-off  between  cost  and  accuracy  in  your  operation? 
Do  you  know  the  trade-off  between  cost  and  turn- 
around time?  Do  you  know  your  customers'  trade-offs 
on  these  items?  An  even  better  question:  does  your 
customer  know  his  trade-offs  on  such  items?  If  not, 
does  your  service  include  helping  bim  to  determine 
them?  I  can  think  of  no  greater  waste  of  resources 
than  giving  your  customers  services  they  really  do  not 
need  and  should  not  have  to  pay  for. 

So  the  message  is:  measurement  man  does  not  live 
by  precision  alone,  but  by  all  the  dimensions  and 
interactions  in  the  measurement  and  calibration  proc- 
ess. If  we  are  to  talk  of  innovation,  we  must  talk 
about  innovation  in  all  phases  of  that  process.  We 
must  not  fall  into  the  trap  of  thinking  that  innovation 
is  always  aimed  at  one  more  decimal  place.  One  can 
easily  see  how  a  standards  laboratory  could  improve 
itself  right  out  of  business  by  pushing  accuracy,  and 
thereby  costs,  in  an  unreasonable  way.  Innovation 
may  be  aimed  at  one  more  decimal  place,  but  it  is 
equally  possible  and  equally  valuable,  under  the  proper 
circumstances,  for  it  to  be  aimed  at  cutting  cost  for  a 


given  accuracy,  at  increasing  the  knowledge  of  the 
uncertainty  in  a  calibration,  or  at  any  one  of  a  number 
of  other  factors. 

To  manage  this  process,  you  must  know  in  a  quanti- 
tative sense  what  the  parameters  of  your  operations 
are,  and  most  especially  the  cost-benefit  relationships 
in  your  operation,  your  customers'  operations,  and  in 
the  interactions  between  you  and  them.  It  is  only  with 
this  sort  of  knowledge  that  you  can  tell  progress  from 
change  and  distinguish  improvement  from  innovation. 

In  this  connection  I  think  that  the  recent  develop- 
ment by  the  National  Bureau  of  Standards  of  the 
Measurement  Analysis  Program  is  a  significant  step 
in  the  right  direction.  This  MAP — the  idea  must  be 
a  good  one  since  it  already  has  an  acronym — belongs 
to  the  next  generation  of  calibration  techniques.  NBS 
is  now  offering  MAP  for  mass  calibrations  and  is 
investigating  the  possibilities  for  calibrations  of  voltage 
standards  and  of  gage  blocks. 

MAP  recognizes  that  the  laboratory  in  possession  of 
a  nicely  calibrated  set  of  weights  has  indeed  nothing 
else  for  certain.  It  is  entirely  possible  to  misuse 
dreadfully  a  well-calibrated  set  of  weights  and  provide 
very  poor  calibration  services.  MAP  is  designed  to 
calibrate,  in  effect,  the  laboratory's  entire  operation, 
to  spot  errors  in  handling,  random  or  systematic 
environmental  effects,  and  so  on. 

Even  more  important,  over  a  period  of  time  the 
MAP  data  give  the  standards  laboratory  a  concise  and 
quantitative  history  of  its  operation  so  that  it  can  spot 
instantly  any  anomalies  which  might  crop  up. 

Now,  you  see,  we  are  beginning  to  develop  a  tool 
which  will  give  the  manager  of  a  standards  laboratory 
the  kind  of  broad  and  continuing  overview  of  his 
operations  that  I  have  been  advocating  here  tonight. 
The  very  facts  that  MAP  is  brand  new,  that  it  is  just 
being  developed  in  a  few  areas,  that  it  is  being  well 
received  by  the  laboratories  which  have  begun  to  use 
it,  are  all  evidence  of  how  little  most  laboratories 
really  know  about  their  own  programs. 

I  am  glad  to  say  that  NBS — one  of  my  departments 
— is  taking  the  lead  in  such  an  important  effort.  I 
know  they  will  continue  to  develop  and  to  expand  MAP 
to  the  benefit  of  the  entire  measurement  system.  I 
think  all  of  you  here  should  be  thinking  in  these  terms 
and  using  such  tools  as  this  to  develop  the  most  com- 
plete picture  possible  of  the  operations  in  your  own 
laboratory,  your  relationship  to  your  customers,  and 
your  place  in  the  measurement  system.  It  is  only  with 
this  kind  of  outlook  that  you  can  reliably  tell  culturally 
induced  innovation  from  true  progress  in  the  measure- 
ment and  calibration  arts. 
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included  included  the  phenomenology  of  beta  transmission  through  plural  media,  the  subtleties  of 
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1.  Introduction — The  Mensuration 
Problem 

The  emphasis  in  gas  turbine  design  for  greater 
efficiencies  and  increased  power/weight  ratios  has  led 
to  the  use  of  higher  operational  pressure  and  rotational 
speeds.  This  has  resulted  in  an  increase  in  the  stresses 
imposed  on  the  power-stage  turbine  blades.  Accord- 
ingly, in  order  to  abate  these  stresses,  it  has  been  re- 
quired to  invest  these  turbine  blades  with  internal 
passageways  to  facilitate  air-cooling  and  improve  their 
metal  temperature  profile.  This  has  been  accomplished 
by  an  investment  casting  process  which  can  be  used  to 
cope  with  the  complex  curvilinear  dimensional  re- 
quirements of  the  blade's  contoured  surfaces.  A  typi- 
cal air-cooled  turbine  blade  casting  is  shown  in 
figure  1. 

As  a  consequence  of  the  effort  to  reduce  blade  metal 
temperatures  by  the  incorporation  of  air-cooling 
passageways,  quality  control  inspection  techniques  had 
to  be  bolstered  to  insure  the  retention  of  the  blade's 
structural  soundness  and  dimensional  accuracy.  Such 
inspection  mainstays  as  x  ray  and  Zyglo  which  were 
well  established  for  determining  flaws  had  to  be  supple- 
mented— for  another  structural  characteristic  had 
soared  into  prominence:  wall  thickness.    What  previ- 


ously had  been  spot-checked  by  a  destructive  section- 
ing technique  had  now  to  be  scrupulously  determined, 
for  if  during  the  investment  casting  process  an  occa- 
sional miscast  occurred  in  the  form  of  an  internal  core 
shift,  so  that  blade  walls  buttressing  one  side  of  the  core 
were  made  too  thin,  blades  with  such  unduly  thin  walls 
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Figure  1.  Typical  castings  measured  for  wall  thickness 
by  radiological  mensuration  technique. 
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might  rupture  under  the  imposed  centrifugal  forces. 
If  one  such  blade  were  mounted  in  a  power  turbine 
wheel  of  62  blades  and  should  fail,  the  resultant  weight 
imbalance  might  cause  the  entire  wheel  to  fail  with 
severe  damage  to  the  engine. 

Several  well-established  mensuration  techniques,  in- 
cluding eddy  currents  and  ultrasonics,  were  adapted 
to  tackle  the  wall-thickness  measurement  problem,  but 
all  were  confounded  by  an  obvious  difficulty:  None  of 
them  could  satisfactorily  measure  the  wall  thickness  at 
the  points  of  highest  stress  concentration — the  linear 
distances  (shown  dashed  in  fig.  2)  from  the  narrow 
ends  of  the  covered  space  approaching  tangency  with 


from  zero  to  a  fixed  maximum  energy  which  uniquely 
characterizes  the  particular  species  of  beta-emitting 
radioisotope.    The  effect  is  the  generation  of  a  con- 
tinuous beta  energy  distribution  having  two  termina:  j 
zero  energy  and  a  maximum  energy.    The  beta  spec-  1 
trum  thus  reflects  a  complex  beta-emission  probability 
distribution,  to  wit:  Each  radioactive  atom  has  a  prob- 
ability that  a  beta  particle  will  be  emitted;  and  addi- 
tionally, each  radioactive  atom  has  a  probability  dis- 
tribution associated  with  the  range  of  possible  beta 
particle  kinetic  energies  that  can  be  emitted.  (See 
fig.  10  in  appendix  for  depiction  of  a  typical  beta  | 
spectrum. ) 


WALL  THICKNESS  MEASUREMENTS  SOUGHT 
ARE  SHOWN  AS  DOTTED  LINES 


BLADE  SECTION 
NO  CORE  SHIFT 


BLADE  SECTION 
WITH  CORE  SHIFT 


Figure  2.  Properly  cast  blade  compared  to  core-shifted  blade. 


the  external  blade  surfaces.  It  became  apparent  that 
current  mensuration  techniques  reliant  upon  macro- 
dispersive  or  echo-sensing  phenomena  could  not  focus 
sufficiently  on  these  wall  thicknesses  to  provide  accurate 
results.  Subsequently,  an  innovative  dial  micrometer 
gage  was  tried,  but  the  narrow  (5/64  in.)  access  holes 
to  the  internal  passageways,  together  with  the  steep 
thickness  gradient  in  the  region  of  the  tangency  points, 
especially  when  core  shifting  was  pronounced,  caused 
difficulties  which  could  not  be  surmounted.  The  con- 
ditions for  resolving  the  problem  were  now  beginning 
to  clarify.  What  was  needed  was  a  mensuration  tech- 
nique which  had  very  high  resolution  capability — a 
technique  which  could  virtually  focus  in  on  the  thin- 
walled  corner  pockets  caused  by  core  shifting. 

2.  Genesis  of  the  Radiological 
Mensuration  Technique 

After  some  rumination,  the  idea  of  using  beta 
transmission  for  wall-thickness  measurement  was  lit 
upon.  The  phenomenology  of  beta  transmission 
through  thick  foils  was  well  established — electrons  or 
positrons  when  given  kinetic  energy  will  issue  in 
random  orientations  from  the  nuclei  of  radioactive 
atoms  undergoing  nuclear-energy  transformations  at 
random  time  intervals.  The  kinetic  energies  imparted 
to  these  charged  particles  are  multivarious,  ranging 


Thus,  beta  particles  of  multivarious  energies  and 
manifold  orientations,  many  travelling  close  to  the 
speed  of  light,  stream  pointillistically  from  the  nuclei 
of  origin  into  the  ambient  matter,  sustaining  energy 
absorption  loss  and  scattering  interactions  with  the 
atomic  electrons  and  atomic  nuclei.  The  interactions 
are  profuse  and  the  path  of  the  beta  particle  is  very 
tortuous — a  path  that  is  the  compounding  of  four 
interaction  phenomena: 

( 1 )  Scattering  by  atomic  electrons  without  beta 
energy  degradation  (scattering)  ; 

(2)  Scattering  by  atomic  electrons  with  beta 
energy  degradation  (ionization)  ; 

(3)  Backscattering  by  atomic  nuclei  without 
beta  energy  degradation  (backscattering)  ; 

(4)  Scattering  by  atomic  nuclei  with  energy 
degradation  resulting  in  transformation  of  beta  energy 
to  electromagnetic  wave  energy  (Bremsstrahlung) . 

Indeed,  a  moderately  energetic  beta  particle  sus- 
tains such  a  multiplicity  of  energy  absorption  loss  and 
scattering  collisions  while  passing  through  a  0.020-in. 
thickness  of  stainless  steel  that  it  resembles  a  wavering, 
weakening  drunk,  who  in  trying  to  toe  the  long  white 
line  to  its  terminus  reaches  the  terminus  by  virtue  of 
the  multiplicity  of  his  random  deviations.  During  his 
reeling  walk,  the  drunk  is  not  apt  to  sober  up,  nor  is 
the  beta  particle  likely  to  convert  its  kinetic  energy  to 
Bremsstrahlung  despite  the  multiplicity  of  collisions 
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t  experiences.    Actually  a  beta  particle  of  1  MeV 
Energy,  which  sustains  103  to  104  collisions  while 
iassing  through  a  0.020-in.  thickness  of  stainless  steel, 
uffers  a  Bremsstrahlung  generation  of  only  a  few 
bercent  of  its  total  energy  loss.    This  fortuity  proves 
o  be  of  cardinal  importance  relative  to  the  efficacy  of 
eta  transmission  as  a  wall  thickness  measurement 
technique,  as  will  be  seen  later. 

As  a  result  of  the  multitudinous  energy  absorptions 
and  diverse  scatterings  it  experiences,  the  effective  path 
ilength  of  the  beta  particle  of  kinetic  energy  E  is  pro- 
Iportional  to  the  actual  path  length  and  is  a  simple 
linear  function  of  E.  Thus  if  2-MeV  monoenergetic 
1  electrons  were  used  for  thickness  measurement,  the 
attenuation  characteristic  would  be  described  by  noting 
the  diminution,  with  increasing  material  thickness,  of 
!the  conical  solid  angle  formed  by  the  pencil  of  rays 
representing  the  range  of  possible  effective  electron 
path  lengths.  Then  the  extent  of  the  conical  solid 
angle  emanating  from  the  point  of  electron  entry  would 
be  constrained  by  the  maximum  effective  path  length 
of  the  2-MeV  electrons.  If  this  were  0.050-inch  in 
steel,  then  the  loci  of  possible  electron  emission  sites 
for  an  absorber  thickness  X  would  be  circular  areas 
irR2  where  R2  =  (0.050) 2  -  X2  which  would  plot 
approximately  as  a  straight  line  on  Cartesian  coor- 
dinates for  a  middle  range  of  X  thickness  values. 

A  formalization  of  the  range  of  the  electron  path 
lengths  is  Feather's  rule,  an  empirical  formulation 
relating  maximum  electron  energy  to  maximum  thick- 
ness penetration  for  material  of  a  given  density.  It 
states  that  for  E  >  0.6  MeV,  the  range  of  the  electrons 
in  the  material  is  given  by 

/?(g/cm2)  =  0.542  E  -  0.133 
where  the  range  R  is  the  maximum  thickness  penetra- 
tion for  the  given  material.    Another  empirical  formu- 
lation is 

#(mg/cm2)  =  412  E1-265  ~  °-n954  ln£, 
which  is  applicable  to  the  range  of  0.01  ^  E  =  2.5  MeV. 

It  is  remarkable  fact  that  both  the  identity  and 
uniqueness  of  the  beta  spectrum  are  preserved  even  as 
beta  particles  issuing  from  the  nuclei  of  origin  are 
gradually  attenuated  by  successive  thicknesses  of 
material.  Thus,  a  beta  spectrum  having  a  maximum 
energy  of  Emax  retains  its  characteristic  form  regard- 
less of  the  fraction  of  its  total  energy  that  has  been 
absorbed  by  matter.  In  other  words,  if  an  absorber 
is  used  to  reduce  the  beta  particle  intensity  to  a  frac- 
tion of  its  original  intensity,  the  beta  particles  stream- 
ing out  of  the  surface  of  the  absorber  have  the  same 
mass  absorption  coefficient  1  (cm2/g)  as  the  betas 
streaming  into  the  absorber.  This  implies  that  the 
average  energy  loss  per  incremental  path  distance 
— AEt/&X  is  independent  of  X  and  is  solely  a  linear 
function  of  the  total  energy  Et;  also,  that  the  proba- 
bility distribution  of  orientations  induced  by  scattering 
and  backscattering  is  independent  of  X.  Thus, 
dEt/dX  =  —kEt  and  Et  —  Eu  e~~ kX ;  but  as  Et  is  equal 
to  the  product  of  the  average  beta  kinetic  energy  and 
number  of  beta  emissions,  or  Et  =  N  •  E&v%,  and  £avg 


is  invariant  since  the  beta  spectrum  does  not  change 
its  proportionality,  we  then  have  Nx  —  N0  e~vx  where 
N0  is  the  initial  number  of  beta  emissions,  Ns  is  the 
number  of  beta  emissions  transmitted  through  absorber 
thickness  X,  and  jx  is  the  linear  attenuation  coefficient. 

A  graphical  display  of  mass  absorption  coefficients 
for  a  few  pure  beta  emitters  and  their  corresponding 
thickness  penetration  values  for  Inconel  having  a 
density  of  7.9  g/cm3  are  given  in  figure  3.  Feather's 
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1  The  mass  absorption  coefficient  for  a  given  absorber  is  the  linear 
attenuation  coefficient  or  transmission  constant  of  the  absorber  divided 
by  its  density. 


MAXIMUM  THICKNESS  PENETRATION  IN  INCONEL 
(AS  GIVEN  BY  FEATHER'S  RULE) 

Figure  3.  The  relationship  of  beta  energy  to  mass  ab- 
sorption coefficient  and  Inconel  plate  thickness. 


rule  is  also  delineated.  The  corresponding  equivalent 
mass  absorption  coefficients  for  electromagnetic  wave- 
form radiation  (gamma  or  x  ray)  are  indicated  ad- 
junctly  to  the  depicted  beta  emitter.  However,  since 
the  authors,  after  a  fairly  thorough  search,  were  unable 
to  find  a  commercially  available  radioisotope  of  half- 
life  greater  than  one  year  which  was  either  a  pure 
x  ray  emitter  (electron  capture  made),  a  pure  gamma 
emitter,  an  x  ray  and  beta,  or  gamma-ray  and  beta  emitter 
with  a  composite  mass  absorption  coefficient  comparable 
to  that  of  any  pure  beta  emitter,  no  identification  of  such 
alternates  is  included.  In  fact,  the  use  of  a  beta- 
gamma  emitter  of  disproportionate  mass  absorption 
coefficient  would  result  in  the  gamma  emission  con- 
tributing a  fairly  constant  background  noise  to  the 
beta's  attenuation  slope.  To  illustrate:  The  only  beta 
emitter  of  any  consequential  halflife  which  is  more 
energetic  than  yttrium  90  is  the  daughter  of  ruthenium 
106.  Ruthenium  106,  a  pure  beta  emitter  of  0.039 
fma!1  with  a  halflife  of  one  year,  decays  to  rhodium 
106,  a  30-s  halflife  daughter  which  has  a  beta  of 
3.53  Emax  with  an  emission  probability  of  0.68  per 
atomic  disintegration  and  gamma  emissions  of  0.513 
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MeV  and  0.625  MeV  with  emission  probabilities  of 
1.00  and  0.53,  respectively.  Since  the  mass  absorption 
coefficients  of  the  0.513  MeV  and  0.624  MeV  gammas 
are  less  than  1/40  that  of  the  3.53  beta,  the  gammas 
would  behave  as  a  significant  noise  factor,  and  thus 
contaminate  the  beta  attenuation  curve.  It  appeared, 
therefore,  that  yttrium  90  loomed  as  the  optimal  choice 
for  measuring  wall  thickness  in  the  range  of  0.015  to 
0.040  in.,  which  was  the  anticipated  dimensional  range 
of  forthcoming  mensuration  problems. 

Strontium  90,  a  pure  beta  emitter  of  0.54  Emax  with 
a  halflife  of  28  yrs,  decays  to  yttrium  90,  a  64-hr 
halflife  virtually  pure  beta  emitter  of  2.27  jBmax  (1.734 
gamma  emission  less  than  0.0002  probability),  which 
in  turn  decays  to  stable  zirconium.  Whereas  the 
strontium  betas  are  stopped  by  a  0.009-in.  stainless 
steel  foil  thickness,  the  more  energetic  yttrium  betas 
are  merely  attenuated  by  0.015-in.  to  0.040-in.  thick- 
nesses. Since  the  attenuation  gradient  for  Inconel 
thickness  in  this  range  is  steep  and  very  nearly  ex- 
ponential, wall  thickness  differentiability  was  readily 
obtainable.  The  attenuation  gradient  may  be  ex- 
pressed as  the  change  in  counting  rate  (customarily 
counts/minute)  per  unit  change  of  material  thickness, 
where  counting  rate  is  the  number  of  discrete  radia- 
tions entering  a  radiation  detector  and  counted  dur- 
ing a  time  interval.  The  attenuation  of  a  pure  beta 
emitter  such  as  yttrium  90  with  increasing  material 
thickness  over  a  significant  fraction  of  the  yttrium  90 
beta  penetration  range  is  closely  approximated  by  a 
straight  line  on  semilogarithmic  paper,  where  counting 
rate  is  the  logarithmic  scale  and  material  thickness 
the  linear  scale. 

From  a  macroscopic  point  of  view,  the  principle  of 
using  beta  transmission  for  measuring  blade  wall 
thickness  at  a  particular  site  appears  simple  and 


straightforward.  A  radioactive  point  source  is  juxta- 
posed beneath  the  blade  wall  site,  and  a  radiation 
detector  positioned  in  proximity  above.  The  number 
of  beta  particles  issuing  from  the  radioactive  source 
and  transmitted  through  the  blade  wall  is  then  de- 
termined as  a  counting  rate  by  the  detector,  and  the 
wall  thickness  determined  by  referral  to  a  calibration 
curve  relating  counting  rate  and  wall  thickness  for 
the  given  counting  geometry.  See  figure  4.  However, 
the  reader  can  appreciate  the  pitfalls  which  can  befall 
the  user  who  is  unaware  of  the  intricate  phenomeno- 
logical  interactions  which  may  be  contributory  to  the 
calibration  curve.  The  previous  discussion  concerned 
only  the  phenomenology  of  beta  transmission  through 
a  single  medium.  If  now  the  problem  is  complicated 
by  the  condition  of  plural  media,  as  is  the  customary 
condition  in  counting  configurations,  whatever  con- 
tributory factors  may  be  present  in  the  single  medium 
will  be  magnified. 

There  are  customarily  seven  media  involved  in  the 
usual  counting  configuration.  Beta  particles  issuing 
from  the  radioactive  source  transmit  sequentially 
through  the  radioactive  source  itself,  through  the 
source  containment,  through  air,  through  the  wall 
thickness,  through  air,  through  the  detector  window, 
and  finally  into  the  detector  sensitive  volume. 

If  the  radioactive  source,  source  containment,  de- 
tector window,  and  detector  sensitive  volume  are 
maintained  without  material  or  spatial  alteration 
throughout  the  series  of  counting  periods,  these  media 
will  merely  represent  collectively  an  absorber  which 
reduces  the  effective  range  of  wall  thickness  measure- 
ment. Clearly,  if  these  media  have  collectively  a  high 
mass  absorption  coefficient,  thickness  measurement  may 
be  precluded  over  a  significant  fraction  of  the  thick- 
ness range. 


Figure  4.  Calibration  curve,  yttrium  90  transmission  versus  Inconel 

plate  thickness. 
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3.  Counting  Geometry  Considerations 

In  the  macro  sense,  the  key  to  obtaining  consistently 
accurate  results  is  the  maintenance  of  constant  count- 
ing geometry  during  the  measurement  sequence. 
Several  subleties  are  implicit  in  this  statement,  since 
the  maintenance  of  constant  counting  geometry  de- 
pends not  only  upon  the  spatial  configuration  of  the 
three  elements — radioactive  probe  (radioactive  source 
and  source  containment),  wall  thickness  to  be  meas- 
ured, and  radiation  detector — but  also  upon  the  spa- 
tial configuration  of  the  wall  thickness  itself.  If  the 
wall  thicknesses  to  be  measured  have  variable  thick- 
ness gradient  profiles  or  do  not  have  the  same  ambient 
matter  distribution,  measurement  errors  may  ensue. 

A  brief  example  will  elucidate  the  factor  of  ambient 
matter  distribution.  If  the  reader  refers  to  figure  1 
and  notes  the  numerous  air-cooling  cuts  along  the 
edge  of  the  turbine  blade,  and  imagines  a  point  source 
of  radioactivity  moving  along  the  interior  corner  of 
the  core  space  in  which  these  holes  terminate,  he  may 
appreciate  the  periodically  ascending  and  descending 
counting  rate  that  would  be  observed  as  the  point 
source  traverses  the  path  length,  and  further  appreci- 
ate that  blades  with  such  cuts  might  give  thickness 
variations  from  0.002  to  0.015  in.  in  excess  of  the 
thickness  readings  for  blades  without  such  cuts.  In- 
deed, this  adventitious  finding  was  used  as  a  calibra- 
tion check  to  ascertain  the  proper  spacing  and  posi- 
tioning of  these  cuts! 

The  turbine  blade  wall  thickness  measurement  prob- 
lem involves  measurements  of  variable  wall  thickness 
profiles,  since  it  is  the  exceptional  case  of  core  shift 
that  produces  excessively  thin  walls.  If  the  reader 
refers  to  figure  2,  he  can  visualize  the  possible  core 
shifts  that  would  create  this  condition. 

The  dimensional  properties  of  the  radioactive  probe 
were  of  paramount  importance  to  the  wall  thickness 
measurements  to  be  undertaken.  It  was  desirable  to 
assure  that  the  radioactive  probe  closely  approximated 
a  point  source,  for  excessive  spatial  extension  of  the 
source  or  of  the  source  containment  would  broaden 
the  surface  area  impinged  by  the  beta  particles  and 
thus  lessen  focusing  ability.  The  two  constraints  on 
the  diminution  of  spatial  extension  were  construction 
capabilities  and  rigidity  considerations. 

4.  Description  of  Radioactive  Probe 

The  radioactive  probe  (see  fig.  5)  was  a  sealed 
source  comprising  essentially  a  stainless  hypodermic 
tube  within  another  stainless  hypodermic  tube,  the 
inner  tube  encapsulating  two  0.002-in.  ceramic  micro- 
spheres 2  perfused  with  a  total  of  three  microcuries  of 
ionically  bound  strontium  90.  The  strontium  90 
microspheres  were  situated  at  the  sealed  tip  end  of 
the  inner  tube,  in  a  pocket  formed  by  the  insertion 
and  securement  of  a  0.0094-in.  piano-wire  plunger  in 
the  inner  hypodermic  tube's  0.010-in.  i.d.  in  a  manner 
so  as  to  snugly  buttress  the  microspheres  against  the 
inner  surface  of  the  tube's  sealed  end.  The  outer 
hypodermic  tube,  with  an  i.d.  of  0.020  in.  and  an  o.d. 

2  The  microspheres  were  fabricated  by  3-M  Company,  St.  Paul, 
Minnesota. 


Figure  5.  Strontium  90  probe. 


of  0.032  in.  tapering  down  to  0.025  in.  at  its  tip,  served 
as  an  overshoe  to  prevent  wear  of  the  0.018-in.  o.d. 
inner  tube  encapsulating  the  strontium  90  source.  If 
and  when  0.0005-in.  wear  of  the  overshoe  occurred,  a 
diurnal  calibration  check,  involving  the  comparison  of 
a  one-minute  test  count  with  the  calibration  standard 
count  of  3500  cpm,  would  be  sufficient  to  discriminate 
0.0005-in.  wear  from  the  wall  thickness  of  0.0065-in. 
thickness  at  the  tapered  radioactive  end. 

After  construction,  the  probe's  beta  emission  profile 
along  its  length  was  determined.  With  the  strontium 
90  probe  in  a  fixed  position  and  a  1.4  rag/cm2  end- 
window  Geiger-Mueller  detector  viewing  it  from  above, 
a  0.015-in.  stainless  steel  foil  was  interposed  between 
detector  and  used  as  a  length  interceptor  coverlet. 
The  foil  was  indexed  in  0.01-in.  increments  to  cover 
successively  greater  lengths  of  the  probe,  and  the 
counting  rate  of  beta  emissions  was  noted  for  each 
position  of  the  foil.  For  probe  length  interceptions  of 
zero,  and  of  0.01  through  0.09  in.  the  counting  rate 
was  steady  at  10,800  ±500  cpm;  however,  when  the 
foil  was  indexed  to  0.01  in.,  the  counting  rate  dropped 
dramatically  to  6450  cpm,  indicating  at  least  partial 
coverage  of  the  strontium  90  source  by  the  foil.  Sub- 
sequent indexing  to  probe  length  interceptions  of 
0.11,  0.12,  0.13  in.,  etc.,  failed  to  alter  significantly 
the  counting  rate,  which  was  sustained  at  the  level  of 
6500  ±300  cpm.  Thus  it  could  be  concluded  that 
the  radioactive  source  was  effectually  contained  with 
the  0.09  to  0.10-in.  interval  of  the  probe  length.  Itera- 
tion of  the  foregoing  procedure  for  other  rotational 
orientations  of  the  probe  established  the  near-radial 
isotropy  of  the  probe's  beta  emission  profile  at  the 
0.09  to  0.10-in.  interval,  there  being  a  maximum  devia- 
tion of  7  percent  in  beta  emission  intensity  attributable 
to  the  minute  variation  in  hypodermic  tube  wall  thick- 
ness girding  the  radioactive  microspheres. 

5.  Phenomenology  of  Beta  Transmission 
Through  Plural  Media 

There  are  two  principal  factors  involved  in  the 
mechanics  of  beta  transmission  through  plural  media: 
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geometric  dispersion  and  attenuation.  While  the 
former  concerns  dispersion  in  space,  or  in-vacuo  dis- 
persion, the  latter  refers  to  the  combined  energy 
absorption  loss  and  scattering  interaction  phenomena 
of  betas  with  the  atomic  electrons  and  nuclei. 

If  the  medium  is  air  and  the  beta  emitter  is  suffi- 
ciently energetic,  attenuation  will  be  negligible  for 
brief  transmission  paths.  Under  such  conditions, 
transmission  through  air  may  be  deemed  geometric 
dispersion.  Consequently,  if  an  yttrium  90  point 
source  opposite  a  flat  surface  is  separated  by  an  air 
gap,  the  interception  of  the  yttrium  90  betas  by  the 
surface  will  be  represented  by  a  probability  density 
distribution  which  is  a  function  of  geometrical  consid- 
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Figure  6.  Chart  guide  for  "Phenomenology  of  beta 
transmission  in  plural  media"  discussion. 

erations  alone.  Referring  to  figure  6,  wherein  a  planar 
representation  of  a  symmetric  counting  geometry  con- 
figuration is  depicted,  we  note  that  if  surface  1-5  were 
assumed  as  being  flat,  the  probability  distribution  of 
betas  along  surfaces  1-3  and  3-5  would  be  precisely 
the  distribution  of  cos  6  where  6  is  the  angular  dis- 
placement of  the  geometric  ray  from  axis  0-3.  In 
extending  this  analysis  to  the  actual  three-dimensional 
counting  geometry  configuration,  the  planar  represen- 
tation is  revolved  about  the  axis  of  symmetry  generat- 
ing circumferences  which  have  probability  values 
corresponding  to  a  cos  0  {A  tan  6)  distribution. 

If  the  constant  A  is  disregarded,  this  distribution 
reduces  to  sin  9.  Since  there  is  symmetry,  the  posi- 
tion of  the  beta  in  the  circumference  is  immaterial 
and  the  sin  6  distribution  can  be  properly  depicted 
as  being  distributed  along  lines  1-  3  and  3-5.  Thus, 
the  probability  of  a  beta  entering  circumference  5 
would  be  greater  than  that  of  beta  entering  circum- 
ference 4,  and  the  probability  of  a  beta  entering  a 
circumference  more  remote  from  the  origin  that  cir- 
cumference 5  would  be  greater  than  that  of  a  beta 
entering  circumference  5,  and  so  forth.  This  con- 
sideration instructively  emphasizes  the  importance  of 


minimizing  the  source  size,  source  containment,  and 
source  distance  from  the  thickness  to  be  measured,  if 
it  is  anticipated  that  the  wall  thickness  profile  may 
vary.  Ideally,  the  radioactive  source  should  be  a 
point  source  contiguous  to  the  point  at  which  the  wall 
thickness  is  to  be  measured.  If  this  is  nearly  accom- 
plished, then  the  procession  of  beta  transmission 
through  the  wall  thickness  will  resemble  the  conical 
dispersion  of  rays  shown  issuing  from  point  2. 

Beta  transmission  through  matter  combines  the 
factors  of  geometric  dispersion  and  attenuation.  Beta 
particles  entering  at  point  2  undergo  so  many  diffuse 
deflections  and  collisions  in  a  0.020-in.  metal  thickness 
that  the  probability  distribution  relating  to  attenuation 
is  merely  a  function  of  the  effective  path  lengths 
travelled  by  the  betas. 

The  factor  of  beta  attenuation,  however,  is  compli- 
cated by  an  ancillary  factor:  Bremsstrahlung. 
Bremsstrahlung  is  electromagnetic  wave  radiation 
which  is  generated  by  the  interaction  of  the  beta's 
wavefront  with  the  coulomb  field  of  an  atomic 
nucleus.  During  this  interaction  any  possible  fraction 
of  the  beta's  energy  can  be  converted  to  electro- 
magnetic wave  radiation.  Thus,  when  the  beta  in- 
tensity is  sufficiently  high,  the  multiplicity  of 
interactions  results  in  the  generation  of  a  Bremsstrah- 
lung spectrum  characterized  by  the  diminution  of 
photon  intensity  with  increasing  photon  energy;  i.e. 
photon  intensity  is  maximized  as  the  photon  energy 
approaches  zero,  and  is  minimized  as  the  photon 
energy  approaches  Emax.  Since  the  probability  of  pro- 
ducing Bremsstrahlung  is  a  function  of  the  number 
and  intensity  of  the  nuclear  coulomb  fields  confront- 
ing the  beta  particles,  the  intensity  of  Bremstrahlung 
generated  will  increase  to  a  maximum  value  for  a 
material  thickness  corresponding  to  the  beta  range. 
Thereafter,  for  greater  material  thicknesses,  Brems- 
strahlung attenuation  will  occur,  albeit  at  a  much 
lesser  rate  than  that  of  beta  attenuation  since  the 
medial  Bremsstrahlung  photon  energy  of  0.2  MeV 
has  a  mass  absorption  coefficient  about  1/30  that  of 
its  parent  beta.  However,  the  Bremsstrahlung  photon 
contribution  to  the  beta  attenuation  curve  will  be  in- 
consequential except  for  thicknesses  approaching  the 
range  since  the  percentage  of  total  beta  energy  con- 
verted to  Bremsstrahlung  is  Z£max/3000  where  Z  is 
the  absorber's  atomic  number,  or  about  2.5  percent 
for  strontium  90-yttrium  90. 

Since  the  Bremsstrahlung  contribution  to  the 
yttrium  90  beta  attenuation  curve  is  insignificant  for 
Inconel  thickness  loss  than  0.040  in,  we  shall  now 
resume  the  analysis  with  the  imposed  constraint  of 
this  thickness  limitation. 

Referring  to  figure  6,  one  notes  the  typical  effective 
path  lengths  depicted  as  originating  from  point  2.  As 
before  the  geometric  dispersion  probability  distribu- 
tion is  sin  0,  but  with  the  complication  that  6  now 
has  a  limiting  value  fixed  by  the  range  of  the  £max 
betas  through  the  medium.  If  we  calculate  the  relative 
probabilities  of  beta  transmission  along  these  path 
lengths,  as  effectuated  by  the  attenuation  factor  using 
e  -i36.5(o.o2o/cos  $)  wnere  136.5  is  the  approximate  linear 
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Table  1.  Calculated  thickness  penetration  probability  values 
versus  experimentally  derived  probability  values 


Relative  probability 

values 

Wall  thickness 

Combined  geometric  Calibration 

dispersion  and  curve 
attenuation  factors  (cpm) 

(inch) 

67° 

V  o-136  5  (  020/cOS  8 ) 

5  =  0° 

sin  8  — 

0.08795 

5400 

0.030 

53 

g-136.5  (.030/cOS  8) 

8  =  0" 

sin  8  — 

0.01564 

980 

0.040 

37° 

2  e-136.5  (.040/cOS  8  ) 

8  =  0" 

sin  0  = 

0.00312 

175 

attenuation  coefficient 3  per  inch  of  yttrium  90  in  a 
medium  of  7.9  g/cm3  density,  the  proportion  of  these 
attenuation  factor  probability  values  will  correspond 
fairly  well  with  the  counts  per  minute  (cpm)  values 
given  by  the  calibration  curve  for  the  thinner  wall 
sections,  but  suffer  progressively  poorer  correspond- 
ence as  the  thickness  increases,  by  dint  of  the  fact  that 
the  geometric  dispersion  conical  solid  angle  is  dimin- 
ishing with  increasing  thickness. 

It  should  be  noted  that  these  attenuation  factor 
probability  values  would  correspond  to  the  calibration 
curve  cpm  values  if  a  pinhole  collimation  window 
had  been  interposed  between  the  measured  wall  thick- 
nesses and  the  detector  during  the  generation  of  the 
calibration  curve. 

Recalculating  the  relative  probability  values  for 
yttrium  90  transmission  through  Inconel  thicknesses 
of  0.020,  0.030,  0.040  in.,  but  now  considering  both 
the  geometric  and  the  dispersion  factors,  and  using  an 
effective  maximum  penetration  path  of  0.0505  in.  (the 
0.057  in.  range  of  yttrium  90  betas  in  Inconel  dimin- 
ished by  the  source  containment  thickness  of  0.0065 
in.),  we  obtain  the  summations  given  in  table  1.  The 
calibration  curve  cpm  values  abstracted  from  figure  4 
are  juxtaposed  to  facilitate  comparison. 

A  collation  of  complete  results  is  given  graphically 
in  figure  7.  The  results  are  normalized.  As  antici- 
pated, the  beta  particle  attenuation  values  calculated 
from  the  derived  analytical  expression  of  the  combined 
geometric  dispersion  and  attenuation  factors  of  an 
uncollimated  counting  geometry  corresponded  much 
more  closely  to  the  calibration  curve  values  which 
were  determined  from  uncollimated  counting  geometry 
measurements  than  do  those  values  calculated  from 
the  derived  analytical  expression  for  a  collimated 
counting  geometry. 

If  these  calculations  are  performed  for  other  se- 
lected points  on  lines  1-3  and  3-5  and  repeated  at 
these  points  for  varying  gradients,  it  becomes  clear 
that  varying  wall  thickness  gradients  can  produce  seri- 

3  Linear  attenuation  coefficients  are  experimentally  derived;  how- 
ever, it  is  Dossible  from  a  consideration  of  first  principles  to  derive 
not  only  the  analytical  expression  of  the  beta  spectrum  but  also  to 
use  that  beta  spectrum  in  coordination  with  interaction-phenomena 
factors  and  count-geometry  factors  to  derive  the  calibration  curve 
analytically.  An  attempt  to  do  this  in  a  simplified  manner  is  presented 
in  the  Appendix. 
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Figure  7.  Comparison  of  calibration  curve  of  figure  4 
with  analytically  derived  calibration  curves. 

ous  measurement  errors  when  the  radioactive  source 
is  not  contiguous  to  the  point  at  which  the  wall  thick- 
ness is  to  be  measured,  but  this  implies  precise  posi- 
tioning, for  if  the  point  source  had  been  placed  at 
point  5  rather  than  at  point  3,  the  wall  thickness  meas- 
ured would  be  somewhat  greater  than  thickness  3-6. 

Once  the  probability  distribution  of  betas  issuing 
from  surface  7—8  has  been  determined,  the  geometric 
dispersion  considerations  are  once  more  undertaken 
for  the  air  path  to  the  detector  window.  Here  again 
important  parameters  loom,  namely:  the  size  of  the 
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detector  window,  its  distance  from  the  surface  of  the 
wall  thickness  facing  the  detector,  and  its  orientation 
with  respect  to  the  wall  thickness. 

The  window  should  be  oriented  to  favor  the  line  wall 
thickness  measurement  sought.  Collimation  of  the 
window  or  reduction  in  window  size  may  be  used  to 
minimize  "thickness  summing"  in  the  region  of  the 
line  wall  thickness;  however,  this  is  contraindicated 
if  sample  positioning  is  not  precise  since  serious  posi- 
tioning errors  may  ensue. 

Some  mention  should  be  made  at  this  point  of  back- 
scattering  since  it  is  often  a  significant  source  of  error. 
As  the  probability  of  backscatter  (deflection  greater 
than  90  degrees,  of  a  beta  by  an  atomic  nucleus)  is 
steeply  ascending  with  diminishing  energy  of  non- 
relativistic  betas,  and  inconsequential  for  relativistic 
betas,  the  following  discussion  will  concern  only  the 
former. 

The  probability  of  backscatter  relative  to  the  possi- 
ble modes  of  interaction  can  be  given  as  0.25  Z2/j84 
where  is  the  ratio  of  beta  velocity  to  the  speed  of 
light  in  the  medium  considered,  while  the  probability 
of  ionizing  collision,  the  principal  attenuation  mode, 
is  given  as  (2  Z/ /34)  In  ( E  V  2  // )  where  /  is  the  average 
ionization  energy  lost  by  the  beta  per  collision  and  E 
is  in  eV  units.  For  Inconel  the  value  of  /  is  about  330 
eV  1/  s  12  Z).  Hence,  the  ratio  of  ionizing  colli- 
sion probability  to  backscatter  probability  for  beta 
energies  less  than  100,000  eV  is  8/Z  In  (£\/2/330)  or 
about  1.7:1  for  0.1  MeV  betas.  Thus,  if  there  are 
surfaces  in  proximity  of  the  radioactive  probe  which 
function  as  beta  backscatter  reflectors,  and  the  count- 
ing geometry  configuration  has  the  detector  viewing 
these  surfaces,  a  significant  alteration  of  the  calibra- 
tion curve  values  may  occur. 

6.  Test  Results 

A  rigorous  performance  test  was  used  to  determine 
the  feasibility  and  precision  of  the  pullulating  radio- 
logical mensuration  technique:  Each  of  24  vanes  was 
to  be  measured  at  24  sites — these  sites  comprising 
four  interior  corners  at  three  cross-sections  at  both 
sides  of  the  vane — with  the  time  allowance  for  meas- 
urement limited  to  15  seconds. 

The  measurements  were  made  by  manual  insertion 
of  a  strontium  90  radioactive  probe,  similar  to  the  one 
described  in  the  text,  which  was  flagged  and  graduated 
to  cue  the  operator  as  to  the  proper  probe  orientation 
and  probe  insertion  depth.  An  unshielded  low-level 
beta  background  counter  (background  less  than  5 
counts/minute  I  was  used  to  determine  the  beta  trans- 
mission rates.   A  typical  vane  is  depicted  in  figure  1. 

It  was  necessary,  of  course,  to  establish  calibration 
curves.  All  24  vanes  were  first  counted  at  each  of 
their  24  sites.  A  6-s  counting  rate  was  deemed  suffi- 
cient to  ensure  good  counting  statistics,  since  the  av- 
erage counting  rate  was  about  600  counts/6  s  and  the 
anticipated  count  differential  per  0.002-in.  thickness 
differential  was  180  counts/6  s  at  a  thickness  of  0.026 
in.,  the  anticipated  wall  thickness  median.  Thus,  if  the 
precision  criterion  was  a  0.002-in  error  at  a  two-sigma 
deviation  ( where  for  the  appertaining  Poisson  distrib- 


ution, sigma  equals  the  equare  root  of  the  total  num- 
ber of  amassed  counts,  the  background  being  negligi- 
ble) a  two-sigma  deviation  would  be  ±2\/600  or  ±48, ' 
a  value  comfortably  within  the  180  count  differential 
per  0.002  in.    Eight  vanes,  those  which  to  a  first  ap-  ! 
proximation  collectively  provided  an  optimal  range  of 
counting  rates  for  establishing  calibration  curves  over 
an  anticipated  span  of  0.015  to  0.040-in.  thicknesses, 
were  recounted  for  1  min,  and  then  sacrificed  for  the  ] 
calibration  curves.    It  was  possible  by  virtue  of  the 
flatness  of  the  vanes,  and  through  careful  positioning 
of  the  detector,  to  resolve  the  24  formative  calibration 
curves,  one  for  each  wall  measurement  site,  to  a  single 
master  calibration  curve. 

The  eight  vanes  culled  for  sacrifice  were  sectioned 
in  four  parts,  and  optical  measurements  taken  of  the 
minimum  line  wall  thicknesses  at  the  tangency  points 
at  the  confluence  of  the  interior  corners  and  internal 
surfaces.  The  precision  of  these  optical  measurements 
had  a  probable  error  of  0.0005  in.  The  sets  of  data  ' 
relating  counts  per  minute  to  wall  thickness  for  each 
of  the  formative  24  calibration  curves  were  then  fed 
to  a  computer  which  was  programmed  to  determine 
the  value  of  the  slope  which  best  fitted  each  set  of 
data.  The  criterion  used  was  the  method  of  least  ! 
squares.  The  slopes  of  each  of  these  curves  deviated 
from  the  slope  of  the  Inconel  attenuation  curve  by  less 
than  10  percent,  the  average  being  5  percent. 

Subsequently,  a  second  computer  program  was  run 
using  as  data  input  the  slope  value  of  the  Inconel  at- 
tenuation curve  and  again  the  method  of  least  squares 
was  used,  this  time  to  determine  the  best-fitting  slope- 
intercept  value  at  a  0.015-in.  thickness.  Having  es- 
tablished the  constants  of  the  equation : 

log  counts  at  thickness  X  =  —(slope  •  thickness  X  I 
+  counts  at  thickness  0.015  in.), 

a  master  calibration  curve  had  now  been  resolved  from 
the  24  formative  calibration  curves,  and  this  curve 
was  used  for  determining  the  wall  thickness  of  the 
remaining  16  blades  from  the  6-s  counts  which  had 
already  been  taken. 


Table  2.  Error 

distribution 

Strontium  90  Gage  Thic 

kness  Measurements 

Error  Frequency 

Thin  Wall 

Medium  Wall 

Thick  Wall 

Error*  (X 10- i  in.)  0.017-0.025 

0.026-0.030 

0.031-0.040 

in. 

in. 

in. 

0-1  97 

93 

23 

m-2  35 

52 

20 

2V2-3  8 

25 

12 

3V2-4  3 

7 

6 

41/2**  0 

0 

3 

5  0 

0 

0 

51/2**  0 

0 

1 

*Deviation  of  strontium  90  gage  measurement  from  10X 
reticuled  optical  measurement. 

**Measurements  were  taken  to  nearest  0.0005  in. 
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All  16  blades  were  then  sectioned  in  four  parts  and 
optical  measurements  taken  of  each  of  the  24  sites. 
A  comparison  was  made  between  optical  measurements 


and  radiological  measurements, 
in  table  2. 


The  results  are  given 


7.  The  Machine 

On  the  basis  of  the  celerity  and  efficiency  with  which 
the  test  measurements  were  dispatched,  interest  was 
stimulated  for  the  design  of  a  prototype  semi-auto- 
matic machine  version  of  the  strontium  90  radio- 


with  the  preset  counts  dialed  into  the  limit  switch  for 
each  of  the  eight  counting  positions. 

If  the  displayed  counts  exceed  the  preset  counts,  in- 
dicating an  overly  thin  wall,  the  reject  lamp  lights, 
the  probe  is  withdrawn,  the  turntable  automatically 
indexed,  the  part  stamped  reject,  the  turntable  re-in- 
dexed, and  the  part  discharged  into  the  reject  bin.  The 
turntable  then  indexes  the  next  blade  into  measure- 
ment position  and  the  cycle  is  repeated.  Accepted 
blades  which  finish  the  cycle  are  simply  discharged 
into  the  accepted  bin. 

The  machine  has  proven  to  be  fast,  highly  accurate, 


Figure  8.  Closeup  of  turntable,  probe  positioning  mechanism,  and  de- 
tector mounts  of  radiological  mensuration  machine. 


logical  mensuration  technique.  Such  a  machine  was 
built  for  Avco  Lycoming  Division  by  the  Accurate  Ma- 
chine Tool  Company,  Cleveland,  Ohio.  An  overall 
view  of  its  moving  elements  is  given  in  figure  8.  The 
machine  can  be  described  briefly  as  follows: 

A  turntable  outrigged  with  four  mounting  fixtures 
is  indexed  sequentially  to  four  stations:  sample  load- 
ing, measurement,  accept-reject  stamping,  and  accept- 
reject  discharge.  The  machine  has  both  manual  and 
automatic  functions.  The  manual  function  merely  in- 
volves loading  the  blades  and  indexing  the  turntable 
to  the  measurement  station.  Then  the  machine  takes 
over. 

In  rapid  sequence  the  probe,  guided  by  servo  motors 
regulating  orientation  in  five  coordinates  {X,  Y,  Z, 
orbital,  and  azimuthal),  is  advanced  to  and  retracted 
from  four  preset  positions  in  the  blade's  interior  pas- 
sageways, each  time  advancing  into  and  pulling  out 
of  a  5/64-in.  diam  port  at  the  blade  tip.  At  each 
position  a  6-s  count  is  taken  by  the  end-window  Geiger- 
Mueller  detectors  viewing  the  convex  and  concave 
walls  of  the  blade.  These  counts  are  displayed  on  two 
Baird-Atomic  530  readouts  and  electronically  compared 


and  reliable.  In  a  1-hr  period,  the  machine  can  di- 
mensionally  check  out  a  hundred  blades  at  eight  meas- 
urement sites  while  maintaining  an  accuracy  within 
0.002  in.  over  95  percent  of  the  time,  for  the  thin- 
walled  blades  it  is  seeking  to  discriminate. 

8.  Appendix.  A  Non-Rigorous  Analytical 
Method  for  Determining  the  Beta 
Transmission  Calibration  Curve 

The  beta  transmission  calibration  curve  which  re- 
lates beta  count-rate  intensity  to  absorber  thickness  is 
derivable  from  the  beta  spectrum.  If  the  beta  spec- 
trum has  been  experimentally  determined  as  by  a 
magnetic  focusing  /3-ray  spectrometer,  or  analytically 
derived  as  from  Fermi's  theory  of  beta  decay  (Ref.  1), 
an  empirical  formulation  such  as  Feather's  rule  can  be 
used  in  conjunction  with  the  previously  discussed  char- 
acteristics of  beta  transmission  through  plural  media 
to  perform  successive  integrations  over  the  portion  of 
the  beta  spectrum  which  can  transmit  through  the  ab- 
sorber thickness  span  of  interest.  In  other  words,  if 
it  were  desired  to  determine  the  beta  transmission  cali- 
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bration  curve  of  yttrium  90  for  Inconel  absorber  thick- 
nesses of  0.015  through  0.040  in.,  one  would  first  de- 
termine the  beta  energy  having  a  0.015-in.  range.  This 
would  then  constitute  the  minimal  beta  energy  required 
for  penetration  through  the  0.015-in.  thickness.  There- 
upon, using  sin  6  to  generate  the  circumferentially 
symmetric  probability  density  distribution  of  betas 
emitted  from  the  surface,  one  calculates  the  sin  6  dis- 
tribution for  each  beta  energy  of  the  beta  spectrum 
capable  of  transmitting  through  0.015-in.  Inconel,  and 
appends  thereto  the  corresponding  beta  emission  prob- 
abilities as  weight  factors.  Having  obtained  a  column 
of  numerics  which  are  summed  up  to  give  a  probability 
intensity  value,  one  repeats  this  process  for  other  thick- 
nesses in  the  span  0.015  to  0.040  in.  In  this  manner 
one  generates  the  probability  intensity  values  of  the 
beta  transmission  curve.  The  justification  of  this 
method  lies  in  the  fact  that  the  beta  spectrum  propor- 
tionation  is  virtually  invariant  even  as  the  beta  ener- 
gies are  being  diminished  by  successive  absorptions. 
Thus  one  can  reconstruct  the  beta  spectrum  with  mono- 
energetic  electrons  and  apply  Feather's  rule  and  the 
sin  6  distribution  to  the  continuum. 


Table  3.  Calculated  beta  transmission 
probability  intensity  values 


Beta 

emis- 
Beta  sion 
Energy   fre-  — 
(MeV)quency  Range  0.020  0.028  0.036  0.044  0.052  0.060  0.0675 

(inch) 


Generated  sin  6  beta  emission 
probability  values*  per 
absorber  thickness  (inch) 


0.5 

4.5 

0.020 

0 

0        0  0 

0 

0 

0.6 

3.3 

.028 

0.792 

0.7 

2.4 

.036 

.833 

0.702 

0.8 

1.5 

.044 

.879 

.772  0.574 

o.y 

0.7 

.052 

.923 

.842    .722  0.533 

1.0 

0.25 

.060 

.940 

.885    .802  .685 

0.500 

i.i 

0.05 

.0675 

.953 

.910    .845  .758 

.838 

0.456 

1.17 

0 

.0722 

0 

0       0  0 

0 

0 

Beta  transmission 
probability 
intensity  value 

6.86  3.70 

1.61 

0.582  0.166  0.023 

Feather's 

absorption  data 

240  112 

18 

19    6.60  1.75 

(relative  intensity 

values) 

*  Numerics  are  weighted  sin  8  values  representing  approximate 
relative  probability  intensity  values  for  variously  energied  RaE 
betas  being  transmitted  through  and  emitted  from  the  surface  of 
different  thicknesses  of  aluminum  absorber.  Each  sin  6  value  is 
the  complement  of  the  cos  9  value  representing  the  ratio  of 
absorber  thickness  to  beta  penetration  range. 

In  the  absorption  experiments  of  RaE  betas  through 
aluminum  foils,  Feather  used  a  thin  mica-windowed 
Geiger-Mueller  detector.  The  RaE  source  was  thinly 
spread  over  a  3-cm  area  closely  matched  to  the  area 
of  the  detector  window.  The  source  was  mounted  in 
a  slight  cavity  in  a  wooden  block  and  positioned  at  a 
distance  of  2  cm  from  the  window. 

In  order  to  give  credence  to  the  this  proposed 


method,  the  authors  used  the  original  data  of  the  RaE 
(bismuth  210)  beta  transmission  curve  through  alumi- 
num absorbers  as  experimentally  determined  by  Feather 
(ref.  2),  in  conjunction  with  the  RaE  beta  spectrum 
experimentally  derived  by  Neary  (ref.  3).  See  figure 
9.  For  the  sake  of  simplicity  and  elucidation,  a  his- 
togramic  area  summing  technique  was  used  instead  of 
integration  to  determine  the  sin  6  distributions.  These 
approximations  were  sufficiently  accurate  as  will  be 
borne  out  later. 

Since  the  maximum  beta  energy  of  RaE  is  1.7  MeV, 
the  maximum  penetration  depth  of  RaE  betas  in  alumi- 
num was  0.0722  in.,  and  this  thickness  was  adopted  as 
the  constraining  effective  path-length  side  of  the  coni- 
cal solid  angles  formed  by  the  beta  transmission  paths 
in  the  considered  span  of  thicknesses  from  0.020  to 
0.060  in.  A  summary  tabulation  of  calculations  is 
given  in  table  3.  To  facilitate  comparison,  some  datum 
point  values  from  Feather's  RaE  beta  transmission 
curve  are  presented  in  the  same  table. 
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Figure  9.  The  beta  spectrum  of  radium  E  [bismuth 
210)  from  Neary  (ref.  3). 

Normalizations  of  the  beta  transmission  curve  de- 
rived in  table  3  and  Feather's  beta  transmission  curve 
are  compared  in  figure  10.  The  reader  will  note  the 
close  correspondence  for  the  thinner  aluminum  foils 
and  the  lesser  correspondence  for  the  thicker  foils. 
This  is  attributable  to  the  absence  of  a  Rremsstrah- 
lung  correction  factor  in  the  analytical  method  exem- 
plified by  table  3.  Since  the  fraction  of  total  beta 
energy  which  is  converted  to  Rremstrahlung  is  ap- 
proximately ZE/3000  or  0.005  for  aluminum  foils 
shielding  RaE  betas,  and  the  beta  intensity  for  an  un- 
shielded RaE  source  is  about  four  times  the  intensity 
for  a  source  shielded  by  0.020-in.  aluminum  foil,  then 
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we  have  as  the  maximum  possible  Bremsstrahlung 
count  contribution  0.005  X  6.86  X  4  =  0.14  count,  as- 
suming that  only  one  Bremsstrahlung  photon  is  pro- 
duced per  beta  particle.  Since  the  source  was  about 
2  cm  from  the  1-cm  diameter  window,  and  the  pre- 
ponderance of  the  Bremsstrahlung  photon  spectrum 
may  be  assumed  as  penetrating  the  aluminum  foil 
thicknesses,  the  fraction  of  Bremstrahlung  entering 
the  detector  had  effectually  two  origins:  Either  the 
Bremstrahlung  photon  was  generated  at  a  point  in 
the  aluminum  foil  medium  which  was  sufficiently  close 
to  the  point  of  beta  entry  into  the  medium  to  con- 
stitute a  possible  intermediate  path  position  of  a  beta 
which  could  pass  through  the  aluminum  foil,  or  the 
photon  was  generated  at  a  point  in  the  aluminum  foil 
medium  or  ambient  matter  too  far  removed  from  the 
point  of  beta  entry  to  constitute  such  an  intermediate 
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path  position.  Clearly  the  conical  solid  angle  of  the 
beta  particles  intercepted  by  the  detector  was  approxi- 
mately the  same  as  that  for  the  Bremsstrahlung  be- 
cause of  the  limited  0.0722-in  maximum  penetration 
distance  of  the  beta.  Thus  since  the  fraction  of  the 
total  conical  solid  angle  intercepted  by  the  detector  was 

detector  window  area  ^  \/\2 
spherical  surface  area 

then  the  Bremsstrahlung  contribution  derived  from 
beta  particles  not  at  possible  intermediate  path  posi- 
tions was  approximately  1/12  of  0.14  or  0.012  count. 
This  value  was  relatively  constant  for  all  absorber 
thicknesses.  In  addition  the  Bremstrahlung  contri- 
bution derived  from  beta  particles  at  possible  inter- 
mediate path  positions  had  a  variable  value  of 

absorber  thickness 


0.012  X 


0.0722 


Figute  10.  Comparison  of  Feather's  curve  depicting 
radium  E  beta  transmission  through  aluminum  foils 
with  analytically  derived  curve. 


Thus,  although  there  would  be  produced  for  a  0.020-in. 
aluminum  absorber  a  total  Bremsstrahlung  contri- 
bution amounting  to  0.012  +  (  0.020/0.0722  X  0.012) 
=  0.015  count,  which  is  negligible  compared  to  6.86 
counts,  the  Bremstrahlung  correction  for  0.052  and 
0.060  thicknesses  was  significantly  0.02  and  0.022 
counts,  thus  raising  the  calculated  count  values  of  0.166 
and  0.023  to  0.186  and  0.045,  respectively.  Accord- 
ingly, the  adjusted  normalized  values  would  be  2.71 
and  0.66.  These  Bremsstrahlung  corrected  values  are 
shown  in  figure  10  as  a  dotted  line. 

If  the  reader  proceeds  to  determine  the  beta  trans- 
mission calibration  curve  by  this  a  priori  method  he 
will  observe  that  the  transmission  curve  is  only  ap- 
proximately a  straight  line  on  semi-logarithmic  paper. 
Clearly  this  preliminary  determination  will  provide 
him  with  a  good  guide  for  performing  accurate  curve 
fitting  of  the  experimentally  derived  data  in  establish- 
ing the  calibration  curve.  Of  course  the  user  must 
take  into  consideration  the  effective  cumulative  ab- 
sorber thickness  due  to  the  source  containment  and 
the  detector  window,  and  adjust  the  abscissal  values 
accordingly.  Furthermore,  he  must  devise  his  count- 
ing geometry  configuration  so  as  to  minimize  back- 
scatter. 
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AN  AUTOMATED  TIME  DOMAIN  INSTRUMENT  TEST  CONSOLE 
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This  paper  describes  a  computer-controlled  system  which  automatically  tests  the  performance  of 
oscilloscopes  and  other  time  domain  instruments.  The  system  utilizes  advances  in  pulse  technology 
to  speed  up  the  calibration;  process  by  replacing  frequency  domain  test  methods  with  time  domain 
test  methods.  Cost  effectiveness  of  this  system  in  terms  of  reduced  man  hours  to  calibrate  an 
instrument,  reduction  of  systematic  errors  due  to  operator  bias,  and  consistent  reporting  of  test 
results  are  described. 

Key  words:  Automatic  measuring  devices;  automatic  performance  test;  computer  control  systems; 
instrument  calibration  procedures;  instrument  performance  testing;  pulse  technology;  time  domain 
testing. 


1 .  Introduction 

The  growth  of  sophistication  of  electronic  systems 
has  been  proceeding  at  a  rapid  rate,  due  largely  to 
inventions  in  electronic  components  and  in  the  science 
of  cybernetics.  These  systems  require  a  high  degree 
of  expertise  on  the  part  of  the  technical  community  to 
maintain  and  to  calibrate  at  a  level  which  will  assure 
measurement  reliability. 

Both  the  quantity  and  complexity  of  the  testing  and 
measuring  equipment  used  by  a  calibration  facility  to 
maintain  these  systems  are  growing  in  direct  propor- 
tion to  the  systems  themselves. 

The  metrology  and  calibration  program  of  every  or- 
ganization is  directly  affected  by  this  increase  in  the 
measurement  requirement  within  its  jurisdiction.  Exact 
quantities  from  which  to  judge  are  not  available,  but 
one  may  reasonably  estimate  that  the  burden  placed 
upon  a  calibration  facility  is  growing  at  an  exponen- 
tial rate. 

The  calibration  laboratories  are  not  only  faced  with 
moving  through  the  shop  the  apparatus  requiring  cali- 
bration or  repair,  but  they  must  also  acquire  and 
evaluate  variables  data  on  each  piece  of  equipment 
handled.  The  calibration  process,  beginning  with  writ- 
ing instrument  calibration  procedures,  then  measuring 
parameters,  recording  the  measurements,  making  ad- 
justments if  necessary,  etc.,  must  be  streamlined  and 
made  more  cost  effective. 

In  addition,  the  enormous  bookkeeping  problems  as- 
sociated with  the  proper  recording  and  processing  of 
calibration  data  and  their  statistics  have  never  been 
adequately  solved.  Metrologists  have  long  been  aware 
of  the  necessity  of  analyzing  variables  data  on  all  indi- 
vidual equipments  requiring  routine  calibration.  The 
potential  gains  resulting  from  such  an  analysis  have 


not  been  realized  because  of  the  difficulty  and  cost  of 
acquiring  and  storing  reliable  data  on  all  the  equip- 
ment involved.  Solutions  must  be  found  for  the  prob- 
lem of  the  growing  calibration  backlog  and  the  as- 
sociated bookkeeping. 

2.  Computer-Controlled  Test  Systems 

Introduction  of  the  minicomputer  has  made  the  use 
of  dedicated  computers  in  areas  of  test  and  measure- 
ment economically  feasible.  Combinations  of  these 
machines  with  programmable  test  equipments  have  re- 
sulted in  the  introduction  of  automatic  test  systems 
which  have  applications  in  calibration  laboratories. 
Implementation  of  these  computer-controlled  test  sys- 
tems in  the  calibration  laboratories  would  automate 
many  of  the  steps  in  the  calibration  process.  For  ex- 
ample, instrument  calibration  procedures  could  be 
written  in  the  form  of  program  test  tapes,  which  are 
more  easily  revised  and  distributed  than  the  present 
manuals;  instrument  performance  parameters  could  be 
tested  more  easily  using  automatic  test  machinery; 
variables  data  could  be  recorded  more  reliably  and 
cheaply  using  automatic  data  logging.  Any  well  de- 
signed automatic  test  system  will  provide  sufficient 
input/output  equipment  to  automatically  load  instru- 
ment test  procedures  and  record  variables  data;  and 
will  contain  a  computer  which  is  easily  interfaced  with 
programmable  equipments  and  has  sufficient  memory 
capacity  to  allow  programming  in  one  of  the  higher 
level  languages. 

A  factor  which  has  not  generally  been  considered, 
however,  is  the  reduction  of  complexity  in  automatic 
test  systems  made  possible  by  the  use  of  time  domain 
test  methods. 
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3.  Performance  Testing  Using 
Pulse  Techniques 

Time  domain  test  methods  are  much  more  readily 
incorporated  into  a  computer-controlled  system  than 
frequency  domain  methods  because  the  elements  needed 
for  stimulus  generation  and  response  measurement  in 
time  domain  are  pulse  generators  and  digital-to-analog 
converters  I  stimulus  )  ;  sample  and  hold  amplifiers  and 
analog-to-digital  converters  (response).  These  ele- 
ments are  natural  computer  peripherals  and  are  quite 
easily  interfaced  to  a  computer  and  packaged  in  com- 
puter architecture.  Further,  they  naturally  fit  into  re- 
flexive control  modes  (self  test).  Frequency  domain 
test  methods  require  oscillators  (or  sweep  generators) 
which  are  programmable  to  many  discrete  frequencies, 
power  amplifiers  programmable  to  many  output  levels 
(stimulus),  AC/DC  converters,  digital  voltmeters,  fre- 
quency counters,  power  meters,  VSWR  meters,  etc. 
(response).  These  elements  represent  highly  complex 
instruments  which  are  costly  and  difficult  to  interface 
into  an  automatic  test  system.  Their  complexity,  in 
turn,  requires  additional  system  downtime  to  assure 
that  the  system  is  in  calibration. 

The  choice  between  time  domain  and  frequency  do- 
main test  methods  is  dictated  by  the  instrument  being 
tested.  There  is  a  natural  tendency  to  reduce  instru- 
ment bandwidth  as  higher  accuracies  are  achieved, 
resulting  in  two  classes  of  test  instruments:  high-accu- 
racy narrow  bandwidth,  and  low-accuracy  wide  band- 
width. 

The  time  domain  testing  approach  is  divided  into 
two  test  categories: 

1.  Transient  response  test — determination  of  instru- 
ment frequency  bandpass. 

2.  Steady  state  test — determination  of  instrument 
sensitivity,  balance  and  gain. 

The  former  technique  utilizes  a  fast  pulse  generator 
to  generate  a  stimulus  and  a  sample  and  hold  amplifier/ 
ADC  to  measure  a  response.  The  latter  method  uses 
a  DAC  to  generate  a  stimulus  and  an  ADC  to  measure 
a  response.  Dividing  the  test  into  these  two  categories 
provides  two  benefits  in  system  design: 

1.  It  is  not  necessary  to  know  the  absolute  amplitude 
of  the  fast  pulse  generator  in  order  to  perform  the 
transient  response  test.  This  is  true  because  the  com- 
puter normalizes  all  measurements  (rise  time,  over- 
shoot, topline  aberrations),  with  respect  to  measured 
pulse  amplitude. 

2.  Steady-state  tests  can  be  performed  at  DC  using 
the  more  accurate  DAC/ADC. 

4.  Comparison  of  Pulse  and  Frequency 
Response  Measurements 

The  performance  of  electron  voltmeters  (EVM's)  is 
usually  checked  by  applying  a  known  sinusoidal  volt- 
age at  a  number  of  discrete  frequencies  and  measuring 
the  output  with  a  calibrated  meter.  An  alternate  to 
this  technique  is  to  apply  a  fast  pulse  to  the  instru- 
ment under  test  and  measure  its  response.  Since  this 
fasl  pulse  i-  composed  oi  a  Hal  frequenc)  spectrum 
ranging  from  zero  cycles  ( DC  I  to  beyond  cutoff  of 


the  instrument,  the  shape  of  the  output  pulse  describes 
the  instrument's  frequency  response.  This  section  de- 
scribes tests  in  which  frequency  and  pulse  responses 
of  an  HP-400H  EVM  were  measured  and  compared  at 
several  settings  of  the  frequency  compensating  net- 
u  i  irks. 

Figure  1  shows  a  block  diagram  of  the  HP-400H 
EVM.  This  instrument  is  composed  of  an  attenuator, 
wide-band  video  amplifier,  full-wave  rectifier,  and  mil- 
liammeter.  A  large  amount  of  negative  feedback  is 
used  around  the  rectifier-amplifier  loop  to  correct  for 
nonlinearities  introduced  by  the  amplifier  and  rectifier. 
The  output  milliammeter  indicates  the  average  value 
of  the  waveform,  and  its  scale  is  calibrated  in  terms 
of  the  rms  value  of  a  sine  wave. 
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FIGURE  1.  Amplifier-rectifier  voltmeter  block  diagram. 


The  input  attenuator  is  divided  into  two  sections. 
The  first  is  a  1000:1  capacitive-resistive  divider  with 
a  conventional  frequency  adjust  (C4).  This  adjust- 
ment is  one  of  the  most  critical  of  all  in  the  instru- 
ment, and  any  maintenance  or  frequency  "tweaking" 
should  not  be  made  unless  its  performance  is  checked. 
The  first  test,  then,  was  to  check  C4. 

Frequency  response  of  the  instrument  in  its  properly 
calibrated  condition  is  shown  in  curve  IF  of  figure  2. 
We  can  see  that  the  response  is  flat  from  10  Hz  to  4 
MHz,  per  specification.  The  time  response,  shown  in 
curve  IT  of  figure  2,  shows  a  standard  step  function. 
By  purposely  misadjusting  C4,  the  frequency  response 
showed  a  break  point  at  1  kHz,  with  the  output  sta- 
bilizing at  80  percent  (at  6  kHz)  of  the  low-frequency 
value  out  to  4  MHz.  The  time  domain  response  cor- 
responding to  this  condition  is  shown  in  curve  2T  of 
figure  2. 

Examination  of  curve  2T  shows  that  the  pulse  has 
a  rising  characteristic  with  time.  Proper  setting  of 
C4  would  produce  a  square  pulse.  The  departure  of 
the  curve  from  a  step  function  is  found  to  be  20  per- 
cent, the  same  as  that  produced  by  frequency  response 
methods.  It  is  interesting  to  note  that,  experimentally, 
proper  setting  could  be  made  in  the  time  domain  much 
more  easily  and  accurately  than  in  the  frequency  do- 
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Figure  2.  Effect  of  misadjusting  C4  on  HP-400H. 


nain.  Note  that  this  effect  occurs  at  long  pulse  lengths, 
ind  has  a  time  constant  of  approximately  100  /xs. 
From  this  we  would  expect  the  "break  point"  in  the 
Frequency  domain  to  be 

—  -  3kHz. 


The  measured  break  point  was  very  close  to  this  value. 

The  second  test  was  to  misadjust  C21,  the  main 
high-frequency  compensator  in  the  feedback  loop.  As 
can  be  seen  in  figure  3,  too  little  capacity  resulted  in 
the  frequency  domain  curve  3F  and  its  time  transform 
curve  3T.  Note  that  30  percent  peaking  occurred  in 
frequency  response.   This  is  vividly  shown  by  the  ex- 
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ponential  ringing  in  the  time  domain  (about  2  cycles 
to  1/e).  When  C21  was  adjusted  too  high,  a  value 
rolloff  occurred  beginning  at  1  MHz  (curve  4F  of  fig. 
3).  Feedback  amplifiers  of  this  type  are  usually  com- 
pensated for  maximum  frequency  response  and,  as 
such,  will  produce  a  small  amount  of  peaking  near  the 
unity  gain  crossover  frequency.  The  peaking  is  re- 
lated to  pulse  response,  phase  margin,  and  damping 
ratio.  In  general,  for  maximum  flat-amplitude  re- 
sponse the  quadratic  damping  factor  will  be  about  0.7. 
This  produces  no  peaking  in  the  frequency  domain, 
but  will  produce  about  5  percent  overshoot  when  the 
EVM  is  subjected  to  the  step  input.  Thus,  optimum 
frequency  response  will  be  achieved  if  the  time  do- 
main check  is  for  5  percent  overshoot. 

Several  other  frequency  compensation  adjustments 
can  be  made  for  individual  voltage  ranges  of  the  test 
instrument.  Tests  using  frequency  and  time  methods 
produced  curves  5  and  6  of  figure  4  for  C14.  These 
curves  are  similar  to  those  produced  by  C21  (fig.  3). 

Discrete  frequencies  normally  used  to  test  this  in- 
strument are  shown  by  the  circles  on  figure  2.  The 
main  amplifier  has  not  been  tested  because  weak  tubes 
will  cause  a  minor  change  at  the  very  low  frequencies 
(10  Hz)  and  at  the  very  high  frequencies  (>1  MHz) 
due  to  a  change  in  gain-phase  margin  of  the  feedback 
system.  These  parameters  are  usually  checked  by  ex- 
tensive frequency  testing  at  the  extremes  of  the  fre- 


quency  range,  the  very  place  where  pulse  testing  is 
most  sensitive. 

Therefore,  we  must  conclude  that  pulse  testing  is  as 
sensitive  as  frequency  testing  in  detecting  misadjust- 
ment,  and  that  pulse  response  measurements  are  much 
faster  and  better  adapted  to  automated  systems  than 
discrete  frequency  measurements. 

5.  Applications 

Tlic  K(i\(r  model  3003  time  domain  test  system  is 
an  interesting  application  of  pulse  test  methods  in  a 
computer  controlled  test  system. 

Referring  to  figure  5,  the  system  consists  of  ten  ma- 
jor assemblies.  A  description  of  the  function  of  each 
follows : 

1.  Operator  Input/Output  Console — Accepts  program 
tapes  and/or  test  console  operator  keyboard  entries 
and  converts  these  data  to  serial  format  for  servicing 
by  the  stored  program  controller.  It  also  converts 
program  status  messages  and  operator  instruction  to 
English  Language  statements  which  are  typed  on  the 
operator  console.  Further,  it  logs  instrument  per- 
formance test  results  on  punched  paper  tape  for  ma- 
chine processing. 

2.  Stored  Program  Controller — Stores  instrument 
test  program,  transmits  program  status  messages  to  the 
test  console  operator,  interprets  and  executes  test  pro- 
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cedure  instructions.  Transmits  programming  com- 
mands to  the  digital  control  unit,  and  compares  test 
results  with  test  procedure  tolerance  limits. 

3.  Digital  Control  Unit — Decodes  programming  com- 
mands from  the  stored  program  controller  and  routes 
them  to  the  proper  interface  unit.  Accepts  digital  data 
from  measurement  units  and  routes  them  to  the  stored 
program  controller. 

4.  Analog-to-Digital  Converter — Converts  analog  volt- 
ages, from  the  programmed  calibration  fixture  and  the 
strobing  voltmeter  to  digital  numbers  which  are  then 
fed  to  the  digital  control  unit  for  processing  by  the 
stored  program  controller. 

5.  Digital- to -Analog  Converter — Converts  digital 
control  unit  to  six  analog  output  voltages.  These  volt- 
ages perform  the  role  of  stimulus  to  the  instrument 
function  generators  and  amplifiers,  and  of  control  in- 
puts to  analog  interface  units. 

6.  Strobing  Voltmeter — Converts  wide  bandwidth  in- 
put signals  to  narrow  band  output  signals  for  digiti- 
zation by  the  analog  to  digital  converter.  Under  pro- 
gram control,  it  strobes  signals  from  the  programmed 
calibration  fixture  at  precise  times  for  measurements 
of  voltage  as  a  function  of  time. 

7.  Programmed  Calibration  Fixture — Under  pro- 
gram control,  it  (1)  conditions  power  supply  voltages 
to  proper  voltage  and  impedance  levels  for  measure- 
ment by  the  analog-to-converter  unit,  (2)  delivers  pre- 
cision pulse  waveforms  to  the  instrument  amplifiers 
for  transient  response  tests,  and  (3)  conditions  analog 
voltages  from  the  D/D  converters  for  amplifier  gain 
tests. 

8.  Programmed  Line  Voltage  Source — Supplies  ac 
power  to  the  instrument  under  test,  and  varies  the 


line  voltage  from  105  to  125  V  under  program  control, 
for  instrument  power  supply  line  regulation  tests. 

9.  Preset  Counter — Synchronizes  the  instrument  un- 
der test  with  the  strobing  voltmeter,  and  services  those 
program  interrupts  which  signal  functional  comple- 
tion. The  counter  unit  provides  a  time  base  for  real 
time  sampling  for  the  100  /xs/cm  and  slower  oscillo- 
scope sweep  rates,  and  serves  as  an  electronic  frequen- 
cy counter  for  measurement  of  time  mark  interval, 
pulse  generator,  repetition  rate,  etc. 

These  equipments,  in  conjunction  with  programmed 
test  routines  (software)  developed  by  EG&G  for  in- 
strument calibration,  constitute  a  test  system  which 
automatically  sequences  the  test  necessary  to  validate 
performance  of  an  instrument  and  which  indicates 
accept/reject  for  each  parameter  in  the  performance 
test  sequence. 

6.  Operational  Procedures 

Introduction  of  automated  test  systems  into  work- 
load areas  requiring  skilled  people  can  lead  to  job  dis- 
satisfaction, particularly  if  the  system  denies  the  tech- 
nician the  feeling  of  accomplishment  which  he  derives 
from  making  a  significant  contribution  toward  some 
goal.  The  engineer  who  reduces  the  worker  to  the 
role  of  feeding  a  machine  errs  in  his  systems  design; 
he  should  recognize  that: 

1.  Computer-controlled  systems  are  capable  of  high- 
speed data  logging,  automatic  equipment  set  up,  and 
computation  (including  comparison  to  tolerances), 
but  are  not  capable  of  making  qualitative  judgements. 

2.  Human  beings  have  slow  responses  and  cannot 
rapidly  make  computations,  but  are  capable  of  quali- 
tative judgements. 
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Systems  designed  for  use  in  calibration  laboratories 
should  delegate  to  the  computer  the  functions  of 
measurement,  computation  and  control,  but  should 
make  the  operator  an  integral  part  of  the  system.  The 
operator  is  relieved  of  the  rather  mundane  tasks  of 
equipment  setup,  data  recording,  repetitive  measure- 
ment comparisons  and  computations,  but  is  the  sole 
judge  of  instrument  overall  quality,  and  is  responsible 
for  basic  parameter  determinations. 

For  example,  in  oscilloscope  testing,  the  operator 
must  judge  CRT  display  quality,  trigger  amplifier 
hysteresis  characteristics,  amplifier  unbalances,  etc. 
The  computer  queries  the  operator  on  these  measure- 
ments and  proceeds  as  directed.  Having  stored 
measurement  tolerances  from  the  operator  acceptances, 
the  computer  can  go  on  to  repetitive  tests. 

An  important  distinction  is  made  herein.  The 
oscilloscope  cathode  ray  tube  is  a  useful  indicator,  but 
was  never  intended  to  be  a  precision  measurement 
device.  Having  established  that  the  CRT  is  working 
properly,  the  system  can  extract  all  other  parameters 
directly  from  amplifier  outputs  (in  fact,  more  ac- 
curately, since  the  CRT  reading  accuracy  is  propor- 
tional to  beam  writing  rate,  focus,  pattern  distortion, 
flicker,  etc. ) . 

Perhaps  the  most  unique  feature  of  the  time  domain 
system  is  the  interface  unit,  which  among  other  things, 
transforms  the  electrical  signals  extracted  at  the  CRT 
plates  to  levels  suitable  for  processing  by  the  strobing 


voltmeter  without  sacrificing  measurement  system  fre- 
quency bandwidth. 

7.  Conclusion 

Criteria  which  make  automatic  test  systems  suitable 
in  calibration  laboratories  have  been  reviewed.  These 
are: 

1.  Input/output  compatibility  with  the  data  report- 
ing system; 

2.  Computer  architecture  and  programmability ; 

3.  Interfaceability   of   programmable   test  equip- 
ments; 

4.  System  complexity  required  to  provide  total 
calibration  workload  capability. 

The  first  three  criteria  are  met  by  any  well-designed 
automatic  test  system.  The  fourth  criterion  can  be 
effectively  met  by  systems  which  utilize  time  domain 
test  methods. 
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1.  Introduction 

The  purpose  of  this  paper  is  to  describe  how  a  com- 
puter-automated system  is  being  used  to  perform  many 
precision  measurements  typically  required  of  a  stand- 
ards laboratory.  The  approach  taken  using  this  auto- 
matic system  often  has  differed  from  classical 
techniques  and  apparatus  normally  used  by  standards 
laboratories.  Several  typical  applications  that  will  he 
discussed  illustrate  the  use  of  an  on-line  instrumenta- 
tion computer  to  perform  real-time  control  of  experi- 


ments, computer-corrected  measurement  and  stimuli, 
unattended  measurement  processes,  and  the  use  of 
alternative  techniques  that  would  be  impractical  with- 
out the  computational  speed  of  the  computer.  Some 
applications  to  be  discussed  will  also  illustrate  how 
rapidly  precision  measurements  can  be  made  and  how 
statistical  techniques  can  be  used  to  increase  further 
the  confidence  in  the  measurement  results.  It  is  not 
the  intent  of  this  paper  to  present  the  full  details  of 
each  application,  but  rather  to  present  an  outline  of  the 
methods  and  apparatus  used  and  the  results  that  have 


Figure  1.  Computer-automated  calibration  system. 
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been  obtained.  The  automatic  system  has  been  applied 
to  intercomparison  of  saturated  standard  cells,  to 
within  ±0.2  /xV,  standard  resistors  to  within  1  ppm, 
resistance  thermometer  calibration  within  0.01  °C,  and 
to  the  calibration  of  micropotentiometers  and  rf  at- 
tenuators. The  general  methods  used  to  obtain  these 
results  will  be  presented,  as  well  as  a  brief  system 
description,  comments  on  the  system  software,  opera- 
tion, and  general  applications. 

2.  System  Description 

The  computer-automated  system  shown  in  figure  1 
is  being  evaluated  for  general  metrology  and  calibra- 
tion applications  in  the  Army's  calibration  system. 
However,  the  system  is  available  commercially  and  is 
representative  of  many  typical  computer  controlled 
instrumentation  systems.  The  extent  of  parameter 
coverage  in  a  single  system  does  make  this  one  some- 
what unusual,  however.  Measurement  and  stimuli 
instrumentation,  all  computer  controlled,  operate  over 
the  frequency  range  from  dc  to  12.4  GHz.  The 
computer  is  a  16K-word  memory,  16-bit-word  machine 
with  an  input/ output  extender  that  houses  the  inter- 
face between  the  computer  and  the  instrumentation.  A 
magnetic  tape  recorder,  photoelectric  tape  reader, 
teleprinter,  and  graphic  terminal  provide  the  primary 
peripheral  equipment  for  handling  programs  and  data. 
The  instrumentation  is  extensive  and  includes  two 
digital  voltmeters,  a  dc  voltage/current  source,  an  ac 
voltage  source,  a  ratio  transformer,  a  wave  analyzer,  a 
frequency  counter,  and  an  X-Y  recorder.  An  auto- 
matic network  analyzer  is  included  as  a  subsystem  for 
making  s  parameter  measurements  of  rf  devices  from 
100  MHz  to  12.4  GHz.  Two  switching  units,  a  200- 
channel  crossbar  scanner  and  a  4-channel  16-output 
distribution  switch,  are  also  included  as  a  program- 
mable portion  of  the  system.  The  software  supplied 
by  the  manufacturer  of  the  system  includes  two  high- 
level  languages,  BASIC  and  FORTRAN.  The  BASIC 
software  package  can  be  used  to  program  the  entire 
system,  both  microwave  and  nonmicrowave  instru- 
ments. Assembly  language  capability  also  exists  and 
has  been  used  for  some  special  applications. 

Before  discussing  the  measurement  applications,  it 
is  appropriate  to  mention  some  of  the  reasons  for  the 
size  (if  this  particular  automated  system.  First,  the 
concept  of  using  a  single,  automatic  facility  to  make 
broad  spectrum  of  measurements  is  being  developed. 
In  this  sense,  the  system  is  multipurpose  and  is  being 
used  in  a  "job  share"  mode.  Measurements  and  ex- 
periments are  brought  to  the  proximity  of  the  machine, 
rather  than  using  several  different  automatic  systems 
at  different  locations.  This  operational  concept  relies 
upon  the  speed  with  which  measurements  may  be  made, 
upon  its  reliability,  and  also  on  certain  automatic 
features  which  make  possible  24-hr  unattended  opera- 
tion, A  second  reason  for  developing  a  single  multi- 
purpose system  is  that  Army  doctrine  requires  that 
calibration  service  be  brought  to  the  user's  site.  This 
mobile  calibration  service  is  at  lower  echelon  than  what 
would  normally  be  considered  "standards  laboratory" 
work.    A  development  project  is  investigating  the  use 


of  computer-automated  equipment  for  mobile  calibra- 
tion operations  in  the  Army.  A  mobile  automatic 
calibration  system  is  presently  being  envisaged  as  a 
multipurpose  measurement  and  calibration  system. 

3.  Applications 
3.1.  Saturated  Standard  Cell  Intercomparisons 

A  group  of  four  saturated  standard  cells  in  an  air- 
bath  has  been  used  in  our  evaluations  of  the  stability 
and  precision  of  the  d-c  voltage  measurement  and 
source  instrumentation.  It  was  originally  intended  to 
have  these  four  cells  periodically  remeasured  at  our 
standards  laboratory  and  use  in  all  subsequent  calcu- 
lations the  individual  cell  emfs  that  had  been  assigned. 
However,  it  was  noted  in  a  simple  cell  intercomparison 
design  that  when  the  emf  differences  between  these 
four  cells  were  measured  using  the  system's  digital 
voltmeter  and  crossbar  scanner,  the  measured  results 
were  in  agreement  with  the  differences  calculated  using 
the  emfs  determined  in  our  standards  laboratory  to  a 
fraction  of  a  microvolt!  These  surprising  results 
prompted  a  redesign  of  the  cell  intercomparison  experi- 
ment based  on  NBS  Technical  Note  430. 

Twelve  channels  of  the  crossbar  scanner  were  con- 
nected to  the  four  cells  so  that  the  12  possible  emf 
differences  between  the  four  cells  could  be  measured 
by  the  system's  digital  voltmeter.  A  BASIC  language 
program  was  prepared  which  measures  each  of  these 
12  differences  and  calculates  the  differences  of  each 
cell  from  the  mean  emf  of  the  group,  the  new  emf  of 
each  cell,  the  deviation  of  each  observation,  and  the 
standard  deviation  of  a  single  observation.  Each 
observation  is  actually  an  average  of  10  voltage  meas- 
urements made  by  the  DVM.  In  the  cell  intercompari- 
son, the  difference  emfs,  which  are  on  the  order  of  1  to 
20  fiY,  are  measured  by  the  DVM  on  its  100-mV 
range,  whose  resolution  in  the  last  window  is  one 
microvolt.  However,  the  average  of  10  or  more  single 
observations  has  been  found  by  direct  test  to  be  correct 
to  0.2  fxV  plus  any  zero  offset.  The  design  of  the  cell 
intercomparisons  does  use  a  left-right  balance  tech- 
nique and  corrects  for  the  zero  offset  in  the  system. 
The  entire  cell  intercomparison  process  is  very  rapidly 
executed,  taking  about  12  s  to  perform.  This  time 
includes  the  120  voltmeter  readings,  crossbar  switch- 
ing, and  the  calculations  mentioned  above. 

The  process  has  been  statistically  studied  over  two 
time  intervals,  one  16  hours  long,  the  other  6  months 
long.  The  primary  intent  of  the  shorter  term  experi- 
ment was  to  study  the  precision  of  the  instrumentation 
in  the  16-hr  period.  The  cell  surveillance  was  per- 
formed, along  with  other  DVM  tests,  every  30  min 
beginning  at  the  close  of  the  normal  workday.  The 
system's  software  clock  was  used  to  time  and  initiate 
the  total  experiment  which  lasted  approximately  5  min. 
The  process  was  automatically  repeated  every  30  min 
until  interrupted  by  personnel  the  next  morning.  The 
results  of  one  such  overnight  cell  intercomparison  are 
shown  in  figure  2.  As  can  be  seen  from  the  control 
charts,  the  standard  deviations  for  the  standard  cell 
intercomparisons  are  approximately  0.3  aiV.  Simi- 
larly, the  upper  and  lower  control  limits  are  0.3  iiV. 
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Figure  2a.  Typical  16-hour  standard  cell  emf. 


Figure  2b.  16— hour  standard  deviation  of 
standard  cell  group. 

The  study  of  the  same  measurement  process  over  a 
6-months  period  was  conducted  by  running  the  stand- 
ard cell  surveillance  program  once  a  day.  Each  day 
on  which  the  program  was  run,  the  data  were  stored 
on  magnetic  tape.  A  separate  analysis  program  wa9 
written  for  accessing  the  data  from  tape  and  calculat- 
ing the  average  standard  deviation,  the  average  emf  of 
each  cell  in  the  period,  upper  and  lower  control  limits, 
etc.  The  program  also  plots  the  above  information  on 
control  charts.    The  results  of  the  longer  term  process 


are  consistent  with  those  of  the  short-term  experiment, 
i.e.,  within  0.3  microvolts. 

3.2.  Standard  Resistor  Intercomparisons 

The  results  obtained  by  using  the  digital  voltmeter 
to  measure  emf  differences  of  1  to  20  pN  suggested 
that  the  same  measuring  system  could  be  used  in  lieu 
of  a  galvanometer  at  the  output  of  a  Wheatstone  bridge 
for  measuring  two  terminal  resistors.  In  this  method, 
no  attempt  is  made  to  balance  the  bridge.  Rather, 
the  input  and  output  voltages  of  an  unbalanced  bridge 
are  measured  by  the  DVM  and  crossbar  scanner  sys- 
tem. For  precision  resistor  intercomparison  on  the 
order  of  1  ppm,  a  substitution  technique  is  used.  Four 
resistors  are  placed  in  mercury-wetted  stands  wired  in 
a  bridge  arrangement.  The  resistance  value  of  only 
one  of  these  resistors  must  be  known  precisely  and 
enters  the  program  as  the  value  of  the  "standard." 
The  precise  values  of  the  remaining  three  resistors  are 
not  critical  in  the  calculations.  The  preliminary  steps 
of  the  program  include  asking  the  operator  "how  many 
unknown  resistors"  does  he  intend  to  measure?  The 
operator  is  asked  to  enter  (via  the  keyboard)  the 
fractional  or  ppm  value  of  the  resistor  being  used  as 
a  standard. 

After  these  preliminary  manual  steps,  the  measure- 
ment sequence  is  initiated  by  the  computer  after  a 
programmed  wait  period.  A  programmable  supply 
applies  voltage  to  the  bridge  network,  the  digital  volt- 
meter is  switched  via  the  crossbar  across  the  bridge 
input,  which  is  measured  ten  times  and  averaged. 
Then  the  crossbar  switches  the  digital  voltmeter  to  the 
bridge  output  and  an  average  of  ten  measurements  is 
obtained.  Next,  the  polarity  of  the  power  supply  is 
reversed  and  the  process  is  repeated.  These  two  sets 
of  measurements  are  subtracted  to  correct  for  small 
thermal  emfs  and  zero  offset  in  the  system.  The 
process  halts  and  a  message  is  displayed  that  instructs 
the  operator  to  "remove  the  standard  and  place  the 
first  unknown  resistor  in  the  stand."  An  empirically 
determined  wait  of  10  s  was  found  to  be  necessary  to 
allow  for  decay  of  transient  emfs  that  arise  when 
resistors  are  placed  on  the  stand.  The  computer  then 
initiates  the  same  measurement  process  described 
above,  to  measure  the  bridge  input  and  output  voltages 
with  the  unknown  resistor  in  the  bridge.  In  less  than 
1  s,  the  ppm  fractional  deviation  from  nominal  is 
printed  out.  The  program  continues  instructing  the 
operator  to  place  unknown  resistors  in  the  mercury 
stand  until  all  are  measured  and  the  corresponding 
answers  are  displayed. 

Two  mathematical  models  have  been  used  to  calcu- 
late the  value  of  the  unknown.  The  first  of  these  can 
be  used  when  the  standard  and  unknown  are  nominally 
equal.  The  input  impedance  of  the  digital  voltmeter 
is  approximated  as  being  infinite  and  the  simple 
expression, 

(Vx  -  Vs) 


Cx  =  Cs  +  4> 


Vin 


is  used  to  calculate  the  fractional  or  ppm  deviation, 
Cx,  of  the  unkonwn  from  the  nominal  value.  Cs  is  the 
ppm  deviation  from  nominal  of  the  standard.  The 
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measured  bridge  output  voltages  for  the  unknown  and 
standard  are  Vx  and  Vs,  respectively.  Vin  is  the  bridge 
input  voltage.  A  general  solution  has  also  been  used. 
Essentially,  it  consists  of  solving  a  system  of  five 
simultaneous  equations  that  describe  the  bridge  net- 
work. The  five  unknown  quantities  are  the  four  cur- 
rents that  flow  through  the  four  resistors  in  the  net- 
work and  the  product  of  the  current  times  the  standard 
lor  unknown)  resistor.  By  solving  the  system  of  equa- 
tions, no  assumptions  are  made  about  impedance  of 
the  digital  voltmeter  and  the  measurements  can  be 
made  over  a  wider  range  of  validity.  Using  either 
method,  there  is  no  delay  that  the  operator  can  per- 
ceive, other  than  the  10-s  delay  mentioned  above. 

Some  typical  results  of  standard  resistor  compari- 
sons are  illustrated  in  figure  3.  Tests  have  been  con- 
ducted to  determine  the  precision  and  resolution  of 
the  method  using  resistors  in  air  at  23  °C  ±0.1  °C. 
The  precision  or  repeatability  of  the  unbalanced  bridge 
technique  has  not  been  studied  statistically  over  a 
long  time  period.  However,  the  range  of  results  is 
generally  less  than  0.5  ppm  when  measuring  100  to 
10000  ft  resistors.  The  accuracy  of  the  difference 
measurement  has  been  established  by  shunting  a 
standard  resistor  with  1  Mft  (±0.01%  tolerance).  In 
summary,  it  has  been  found  that  the  combined  effect 
of  systematic  and  random  uncertainties  is  less  than  1 
ppm  for  resistor  intercomparisons. 

3.3.  Resistance  Thermometer  Calibrations 

The  redetermination  of  the  constants  for  a  platinum 
resistance  thermometer  is  a  tedious  time-consuming 
task  in  most  laboratories.  An  approach  for  automat- 
ing this  task  was  selected  which  was  intended  to  yield 
results  adequate  for  most  applications.  The  goal  was 
to  automatically  redetermine  the  thermometer  constants 
and  calculate  a  new  table  of  temperature  and  resistance 
ratios  so  that  the  thermometer  could  be  used  to  make 
measurements  of  temperatures  with  uncertainties  not 
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Figure  3.  Resistor  inter  comparison  output. 


exceeding  0.01  degrees  Celsius.  The  approach  which 
was  selected  is  to  use  a  standard  and  test  thermometer 
in  thermal  equilibrium  with  a  liquid  bath  or  other 
gradient-free  environment.  The  resistance  of  these 
two  thermometers  is  measured  at  three  or  more  tem- 
peratures, and  each  temperature  calculated.  From 
these  data,  it  is  a  comparatively  simple  matter  to 
calculate  the  least  squares  coefficients  of  a  second 
degree  polynomial.  In  turn,  these  coefficients  can  be 
used  to  generate  a  new  table  of  resistance  or  resistance 
ratio  as  a  function  of  temperature. 

The  system's  digital  voltmeter  and  crossbar  scanner 
are  used  to  measure  the  voltages  across  the  standard 
and  unknown  or  test  thermometer,  and  also  the  voltage 
across  a  standard  resistor.  The  resistor  and  two  ther- 
mometers are  connected  in  series  and  2  mA  thermome- 
ter current  is  provided  by  a  programmable  constant 
current  source  which  is  stable  to  10  ppm  per  day.  The 
resolution  and  linearity  of  the  digital  voltmeter  are 
adequate  for  determining  the  resistance  ratios  of  the 
two  thermometers  with  respect  to  the  standard  resistor. 
For  example,  the  linearity  of  the  digital  voltmeter  has 
been  found  by  direct  test  to  be  better  than  10  ppm. 
This  amounts  to  an  equivalent  temperature  error  of 
less  than  0.0005  °C.  The  resolution  of  the  digital 
voltmeter  on  its  100  mV  range  is  limited  to  1  fjN  for 
a  single  observation  which  corresponds  to  a  tempera- 
ture resolution  of  0.005  °C.  By  averaging  ten  or 
more  digital  voltmeter  observations,  a  resolution  ap- 
proaching 0.001  °C  has  been  achieved. 

A  BASIC  language  program  has  been  prepared 
which  accomplishes  the  calibration  of  one  platinum 
resistance  thermometer  with  reference  to  a  standard 
thermometer.  Figure  4  illustrates  the  instructions, 
data,  and  results  as  displayed  on  the  system's  graphic 
display  terminal.  In  this  example,  calibration  is  per- 
formed at  the  ice  point,  room  temperature,  and  70 
degrees.  The  software  clock  is  used  to  program  a 
5-min  wait  period  to  allow  each  thermometer  to  reach 
thermal  equilibrium  with  the  bath.  After  this  5-min 
wait,  the  resistor  and  two  thermometer  voltages  are 
automatically  measured  and  stored  in  memory.  The 
operator  is  instructed  to  place  thermometers  in  the 
next  bath.  The  process  is  repeated  until  the  third  set 
of  data  is  obtained.  The  temperature  is  then  calcu- 
lated from  the  measurement  data  for  the  standard  re- 
sistance thermometer  and  its  constants  which  have 
been  previously  included  in  the  program.  The  meas- 
urement data,  temperature,  and  resistance  of  the  un- 
known thermometer  are  listed.  A  solution  for  three 
thermometer  constants,  the  zero  resistance,  the  co- 
efficient for  a  linear  temperature  term,  and  a  coeffi- 
cient for  a  quadratic  temperature  term,  is  found. 
These  constants  are  used  to  generate  a  table  of 
resistance  ratio  and  resistance  as  a  function  of  tem- 
perature for  the  unknown  thermometer.  In  the  ex- 
ample shown  in  figure  4,  the  table  has  been  generated 
in  5-deg  increments,  which  of  course  is  an  arbitrary 
choice.    Any  temperature  interval  may  be  chosen. 

The  accuracy  of  the  results  has  been  verified  by 
intercomparing  thermometers  each  of  which  have  man- 
ufacturer's or  NBS  calibration  reports.  In  general, 
the  goal  of  ±0.01  degrees  uncertainty  appears  to  have 
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Figure  4.  Resistance  thermometer  output. 

been  attained  in  the  range  of  0  to  100  degrees  Celsius, 
though  the  process  is  still  under  study  to  determine 
its  actual  limit.  There  is  no  reason  that  the  program 
and  apparatus  cannot  be  expanded  to  calibrate  more 
than  one  test  or  unknown  thermometer  at  a  time.  Core 
storage,  analog  switching,  and  the  constant  current 
source  are  adequate  to  handle  10  or  more  thermome- 
ters. The  practical  limit  of  the  number  of  thermome- 
ters that  could  be  tested  in  one  15-min  period  is  felt 
to  be  the  size  of  the  gradient-free  temperature  baths 
that  would  be  suitable. 

3.4.  Rf  Applications 

Other  applications  of  this  computer-automated  sys- 
tem have  included  rf  measurements  from  100  MHz 
to  12.4  GHz  using  the  Network  Analyzer  subsystem. 
The  Network  Analyzer  has  been  described  in  the  litera- 
ture [1,  2]1  and  will  not  be  repeated  here.  The  Net- 
work Analyzer  has  been  experimentally  evaluated  by 
measuring  rf  devices  that  had  been  previously  meas- 
ured by  NBS  or  our  own  standards  laboratory.  Re- 
sults of  such  measurements  have  more  than  substan- 
tiated the  accuracy  claims  of  the  manufacturer.  A 
combination  of  the  rf  and  dc  system  instrumentation 
is  being  used  to  determine  the  ac— dc  differences  of  rf 
micropotentiometers  to  better  than  1  percent  relative 
to  a  standard  micropotentiometer. 

4.  System  Software 

The  BASIC  language  compiler  supplied  with  this 
system  is  similar  to  the  BASIC  language  used  in  com- 

1Figures  in  brackets  indicate  the  literature  references  at  the  end  of 
this  paper. 


mercial  time-sharing  computer  services.  It  provides 
conversational-style,  on-line  program  modification  in 
the  same  way.  This  is  commonly  referred  to  as  an 
interpretive  compiler.  The  BASIC  compiler  for  this 
system  allows  external  subroutines  to  be  accessed  by 
a  CALL  statement.  Such  subroutines  or  instrument 
drivers  were  provided  by  the  manufacturer  to  supply 
data  to,  control,  and  read  data  from  the  measurement 
and  control  instruments  in  the  system. 

This  interpretive  or  on-line  mode,  while  expensive 
in  terms  of  computer  core  (requiring  approximately 
6K  of  core),  has  been  found  to  be  convenient  and 
satisfactory.  The  measurement  application  programs 
discussed  in  this  paper  all  were  prepared  in  the  BASIC 
language  by  personnel  with  no  previous  programming 
experience.  No  BASIC  language  application  programs 
were  provided  by  the  system  manufacturer.  The  time 
required  to  prepare  and  debug  a  measurement  pro- 
gram varies,  of  course,  with  its  complexity.  The  pro- 
gram using  the  simple  model  for  the  unbalanced 
Wheatstone  bridge  was  prepared  in  approximately 
5  hr.  The  standard  cell  surveillance  measurements 
program  was  prepared  in  less  than  one  day.  However, 
the  refinements  and  experimentation  with  a  working 
program  before  it  can  be  regarded  as  finished  have 
taken  several  weeks  to  a  month. 

5.  Conclusion 

The  measurement  of  emf,  resistance,  and  tempera- 
ture are  common  problems  of  most  standards  labora- 
tories. The  automated  apparatus  and  methods  de- 
scribed above  do  not  necessarily  yield  results  that 
represent  the  ultimate  with  respect  to  precision  and 
total  measurement  uncertainty.  However,  the  measure- 
ment results  are  satisfactory  for  the  majority  of  cali- 
brations performed  by  industrial  and  Department  of 
Defense  standards  laboratories.  The  applications  dis- 
cussed in  this  paper  were  chosen  to  illustrate  how 
little  or  no  measurement  uncertainty  need  be  sacrificed 
when  automating  a  measurement  process  to  increase 
productivity  and  reduce  skill  levels. 
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Ordinarily  a  great  deal  of  time  is  spent  in  calibrating  bolometer  mounts.  Manual  bridges  require 
considerable  care  for  proper  balances,  bias  adjustments,  and  calibration  procedures.  New  instruments 
have  been  designed  to  automate  many  of  the  manual  operations.  These  instruments  are  self-balancing 
and  maintain  precise  power  levels  with  minimum  adjustments.  Due  to  the  fact  that  in  previous  setups 
a  series  of  measurements  had  to  be  made  in  order  to  average  out  errors  caused  by  temperature 
changes,  the  new  automated  system  described  in  this  paper  can  save  about  90  percent  in  measurement 
time.    In  addition,  a  less  skilled  operator  can  perform  the  measurement  with  the  same  accuracy. 

Key  words:  Automated  dc  substitution;  bolometer  calibration;  radiofrequency  power  measurement; 
self-balancing  Wheatstone  bridge. 


1.  Introduction 

This  paper  presents  a  new  technique  in  making  rf 
(radiofrequency)  power  measurements  which  results 
in  an  overall  cost  reduction  in  the  calibration  of 
bolometer  mounts.  Employing  this  technique  permits 
a  calibration  accuracy  of  better  than  99  percent.  The 
range  of  power  involved  is  from  0.5  to  10  mW  over 
the  frequency  range  100  MHz  to  12.4  GHz  for  coaxial 
systems  and  2.6  to  12.4  GHz  for  waveguide  systems. 

Power  measurements  of  this  type  are  currently 
needed  in  (1)  primary  standards  laboratories,  (2) 
secondary  standards  laboratories,  (3)  inspection  sta- 
tions, and  (4)  production  testing  facilities. 

The  measurement  approach  is  known  as  dc  substitu- 
tion. It  uses  a  sensitive  element  (a  bolometer)  whose 
electrical  resistance  varies  with  temperature.  This 
element  is  placed  in  a  Wheatstone  bridge  circuit  as 
shown  in  figure  1.  When  the  proper  amount  of  dc 
power  is  supplied  to  the  bridge,  the  bolometer  element 
heats  to  the  proper  temperature  to  make  its  resistance 
equal  to  that  of  the  other  three  resistors,  and  the  indi- 


Figure  1.  Dc  substitution  method,  assuming  unity 
calibration  factor  for  bolometer. 


cator  will  show  a  null.  If  we  now  supply  an  additional 
rf  power  to  the  bolometer  element,  it  will  heat  up  to  a 
higher  temperature,  thus  changing  its  resistance  and 
unbalancing  the  bridge.  However,  if  we  also  decrease 
the  dc  power  to  the  bridge  by  an  amount  equal  to  the 
applied  rf  power,  the  temperature  of  the  bolometer 
element  will  remain  the  same.  This  keeps  the  bridge 
in  balance,  and  the  indicator  nulled.  The  amount  of 
applied  rf  power  is  assumed  equal  to  the  difference 
of  the  dc  power  before  and  after  applying  the  rf 
power : 

rf  =  dci  —  dc2  =  dc  substituted. 

This  method  of  measurement  by  using  dc  substitu- 
tion has  proven  to  be  one  of  the  most  accurate  and  is 
the  basis  of  most  of  the  existing  measurement  setups. 
However,  there  is  one  important  consideration:  not  all 
of  the  rf  power  incident  on  the  bolometer  mount 
reaches  the  bolometer  element.  Therefore  the  actually 
applied  rf  power  is  not  exactly  equal  to  the  dc  substi- 
tuted power.  A  small  correction  factor,  called  the 
"calibration  factor,"  is  needed: 

_  .  T.      dc  substituted  power  _  P&Q 

Calibration  factor  K  —  — =— : — —  75—  . 

rt  incident  power  rlf 

In  order  to  determine  the  calibration  factor  of  an 
unknown  test  bolometer  mount  (i.e.,  to  calibrate  a  test 
bolometer  mount) ,  we  must  compare  it  with  a  standard 
bolometer  mount.  The  procedure  is  as  follows  (see 
fig.  2).  Since  we  can  accurately  measure  the  dc 
substituted  power  (P",ic)  of  a  test  bolometer  mount, 
its  calibration  factor  (KT)  can  be  determined  if  we 
can  also  accurately  determine  the  rf  power  incident 
upon  the  bolometer  mount.  This  rf  power  (P'rr)  is 
calculated  by  measuring  the  dc  substituted  power 
(P'dc)  in  a  standard  bolometer  mount  and  knowing 
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its  calibration  factor  (Ks)  with  an  accuracy  traceable 
to  the  National  Bureau  of  Standards:  P'rt  =  P'dc/Ks. 

The  essence  of  the  calibration  is  therefore  to  calcu- 
late accurately  and  conveniently  the  rf  power  incident 
upon  the  test  bolometer  mount. 

2.  Previous  Method 

The  method  previously  used  to  calculate  this  rf 
power  required  a  laborious  procedure  to  determine 
first  the  dc  substituted  power  (P',iC)  to  the  standard 
bolometer  mount.  The  essential  steps  employed  in 
this  procedure  were  as  follows: 

1.  Supply  d-c  voltage  to  the  standard  and  manually 
adjust  the  voltage  until  the  bridge  is  nulled.  This  is 
accomplished  by  adjusting  a  variable  potentiometer. 

2.  Compare  the  dc  voltage  applied  to  the  bridge 
with  an  accurate  dc  voltage  calibrated  with  a  precision 
potentiometer. 

3.  Read  the  dc  voltage  volue  E0. 

4.  Apply  rf  power  to  the  standard  and  repeat  step  1. 

5.  With  an  external  potentiometer,  measure  the 
change  in  dc  voltage  to  the  bridge  by  comparison  with 
the  voltage  EQ. 

6.  Read  the  change  in  d-c  voltage  /\E. 
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Figure  2.  Determination  of  calibration  factor  KT: 
Calculate 

P'i  f  —  P'dc/Ks,  Kt  =  P",i,./P  ,-f  • 


7.  Calculate  the  dc  power  using  the  formula 

P'dc  =  0.025/?o  (2E0  -  AE)  AE  raW 
(where  R0  is  the  bridge  resistance) 

8.  Calculate  the  unknown  rf  power  by  using  the 
formula 

P'rt 


P' 

*  dc 


calibration  factor  of  standard  bolometer  mount 

The  total  time  required  for  an  experienced  operator 
to  make  the  measurement  by  following  these  steps  is 
about  10  min.  However,  during  the  relatively  long 
time  required  to  perform  these  steps,  the  room  tem- 
perature is  unlikely  to  remain  constant,  and  this  will 
introduce  an  error  in  the  measurement.  Additional 
errors  will  also  be  involved  if  during  the  measurement 
time  the  rf  power  source  varies  in  magnitude  and 
frequency.  In  order  to  reduce  the  effect  of  these 
errors,  the  measurements  must  be  repeated  from  three 
to  five  times.  Thus  the  total  time  required  to  make 
an  accurate  (better  than  99%)  measurement  is  usually 
about  30  min  at  best. 

3.  New  Method1 

A  new  instrument  developed  by  Weinschel  Engi- 
neering, called  the  precision  rf  power  level  controller, 
greatly  simplifies  and  automates  the  dc  substituted 
power  measurement  we  previously  described.  The 
salient  features  of  its  design  are  as  follows: 

(a)  provision  for  about  30  mW  of  dc  power  to  the 
standard  bolometer  mount; 

(b)  decrease  of  dc  substituted  power  by  accurate 
fixed  amounts,  0.5,  1.0,  5.0,  or  10.0  mW. 

(c)  through  an  automatic  leveling  loop,  the  rf 
power  supplied  to  the  bolometer  mount  is  forced  to 
remain  equal  to  the  preselected  dc  substituted  power, 
i.e.,  0.5,  1.0,  5.0,  or  10.0  mW. 


1  Weinschel  Engineering  Application  Note  12,  Calibration  of  bolom- 
eter mounts  with  leveled  power,  Form  867-3/70. 


Figure  3.  Measurement  setup  using  new  method. 
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Because  of  these  design  features,  the  measurement  is 
now  simplified,  and  involves  only  the  following  steps: 

1.  Supply  d-c  voltage  and  manually  adjust  it  until 
the  bridge  is  nulled.  This  step  of  the  procedure  has 
been  greatly  simplified. 

2.  Select  desired  decrease  of  power  level  (0.5,  1.0, 
5.0,  or  10.0  mW,  to  within  ±  0.1%  +  1  MW)  by 
setting  a  switch. 

3.  Apply  rf  power,  which  will  automatically  be 
equal  to  the  selected  power  of  step  2. 

4.  Calculate  the  unknown  rf  power  using  the 
formula 

pr    _  selected  dc  substituted  power 

rf      calibration  factor  of  standard  bolometer  mount 


The  total  time  required  for  this  new  method  is  about 
3  min,  providing  a  saving  of  90  percent  of  the  time 
previously  needed.  To  obtain  other  desired  power 
levels  for  dc  substitution,  external  resistors  may  be 
added  to  the  precision  rf  power  level  controller. 
Figure  3  shows  a  complete  measurement  setup  using 
the  new  method  described  in  this  paper. 

4.  Comparison  of  Methods 

The  previous  method  required  about  30  min  by  an 
experienced  operator.  The  new  method  requires  only 
3  min  for  the  same  measurement  for  the  same  accuracy, 
and  its  simplicity  allows  a  less  experienced  operator 
to  make  the  measurement.  The  savings  in  time  and 
costs  for  the  measurement  are  obvious. 
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SESSION  2:  LABORATORY  MANAGEMENT  AND  OPERATIONS  REPORTS 

Chairman:  F.  J.  Dyce 
Martin  Marietta  Company,  Orlando,  Fla.  32805 


MEASUREMENT  COMPARISON  PROGRAMS 

Herbert  S.  Ingraham,  Jr. 

RCA  Corporation,  Defense  Electronic  Products,  Moorestown,  N.J.  08057 

The  measurement  comparison  program  of  1965  circulated  unknowns  in  pairs,  allowing  use  of 
Youden  diagrams  to  distinguish  systematic  from  random  error.  In  1969,  two  pairs  were  sent  out 
several  months  apart,  allowing  check  of  the  correlation  effect;  a  round  robin  was  started  on  accelera- 
tion; a  24-inch  end  standard  was  included  in  the  physical  package. 

Key  words:  Accelerometer  round  robin;  computer  program  for  Youden  diagrams;  correlation  effect; 
end  standard  of  length. 


Innovation  in  metrology  as  a  key  to  progress  is  the 
theme  of  our  Conference.  Certainly  the  establishment 
of  a  Measurement  Comparison  Program  of  national 
scope  meets  the  criterion  for  innovation.  The  basic 
program  is  now  a  historical  innovation.  The  current 
question,  to  paraphrase  another,  is,  "What  have  you 
innovated  for  me  lately?" 

To  determine  the  changes  which  constitute  innova- 
tion in  today's  Measurement  Comparison  Program,  we 
must  compare  the  program  as  it  stood  at  the  1968 
Standards  Laboratory  Conference  and  what  useful 
changes  have  taken  place  since  that  time. 

The  concept  of  circulating  measurement  unknowns 
in  pairs  was,  in  itself,  a  significant  innovation  (in 
1965)  in  the  Comparison  Program.  This  allowed  the 
use  of  the  Youden  diagram  concept  to  separate,  to  a 
high  degree,  systematic  error  from  random  error  in 
the  calibration  and  measurement  process.  This  was  a 
first  in  the  measurement  comparison  process  and  was 
an  extremely  valuable  step. 

It  was  felt,  however,  that  measurement  of  a  pair  of 
unknowns  at  nearly  the  same  point  in  time  left  some- 
thing to  be  desired.  Laboratories  traditionally  repeat 
the  same  type  of  measurement  several  times  a  year, 
and  it  was  felt  that,  if  both  items  of  a  pair  were  meas- 
ured at  the  same  time,  random  errors  would  tend  to 
be  masked  by  an  undesirable  "correlation  effect." 

To  obviate  this  correlation  effect,  we  are  currently 
splitting  our  "pairs"  so  that  they  are  measured  at 
different  times,  separated  by  at  least  a  month.  When 
the  results  of  this  program  are  analyzed,  we  would 
expect  to  have  a  truer  picture  of  a  laboratory's  meas- 


urement capability  than  was  possible  before,  particu- 
larly if  the  correlation  effect  was  significant. 

One  such  split  has  already  been  completed  in  the 
mass  comparison  and  will  be  reported  by  Lloyd  B. 
Macurdy.  These  data,  and  data  from  other  programs 
where  the  split  pair  technique  can  be  applied,  should 
provide  a  base  for  evaluation  of  the  significance  of 
correlation  effect. 

Other  program  changes  since  1968  include  the  ex- 
pansion of  the  Program  to  other  areas  of  metrology 
than  were  previously  included.  These  include  the 
addition  of  a  long  (24-in)  end  standard  in  the  physical 
package,  the  separate  circulation  of  the  mass  packages, 
and,  more  recently,  the  start  of  the  first  round  robin 
comparison  in  acceleration,  currently  circulating  to  20 
participants.  (The  acceleration  comparison  is  the  first 
in  which  the  pairs  of  unknowns  were  deliberately 
separated  to  evaluate  the  correlation  effect.) 

In  the  last  major  round  robin  program,  all  data 
reduction  and  analysis  were  accomplished  by  the  route 
supervisors.  This  was  a  lengthy  and  burdensome 
process,  accomplished  only  through  the  dedication  of 
the  route  supervisors.  Now  the  task  of  data  reduction 
and  analysis  has  been  assumed  by  the  National  Bureau 
of  Standards,  and  they  have  added  a  certain  amount 
of  innovation. 

Primary  to  this  effort  has  been  the  development  of 
a  plotting  program  for  the  CalComp  plotter  to  gen- 
erate the  Youden  diagram  directly  from  the  data  and 
to  obviate  the  manual  plotting  done  before.  In  addi- 
tion to  this,  a  handbook  has  been  developed  to  assist 
in  the  analysis  of  the  Youden  plot,  for  use  by  partici- 
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pants  in  analysis  of  their  results.  This  plotting 
program  can  be  made  available  to  those  who  are 
conducting  measurement  comparison  programs,  either 
as  an  intercompany  program  or  as  an  area  program, 
and  who  wish  to  use  the  Youden  presentation  for  data 
analysis  and  have  a  CalComp  plotter  available. 

The  value  of  the  Measurement  Comparison  Program 
can  be  improved  through  innovation  in  the  future. 


Other  comparison  techniques,  currently  under  consid- 
eration, will  result  in  pilot  comparisons  of  limited 
scope  to  aid  in  evaluation  of  the  techniques.  The  most 
useful  of  these  will  be  incorporated  into  the  Program 
to  provide  the  laboratories  of  the  metrology  com- 
munity with  the  most  effective  method  of  assessing  the 
contribution  of  their  own  innovations  to  their 
measurement  capability. 
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REPORT  ON  THE  1968-69  MASS  MEASUREMENT 
AGREEMENT  COMPARISON  ROUND  ROBIN 

Lloyd  B.  Macurdy 

Staff  Mctrologist,  Mettler  Instrument  Corporation,  Princeton,  N.J.  08540 

Seventeen  laboratories  took  part  in  the  round  robin  measurements  of  the  NCSL  Mass  Comparison 
packages  during  1968  and  1969.  In  order  to  speed  up  the  measurements  the  laboratories  were 
divided  into  two  groups.  The  weights  were  calibrated  by  the  NBS  Mass  Laboratory  at  the  beginning 
and  again  at  the  end,  and  Mettler  Instrument  Corporation  measured  all  sets  near  the  middle  of  the 
series.  Two  sets  of  weights  were  sent  to  each  laboratory  at  times  separated  by  two  months  or  more 
in  order  to  provide  data  on  errors  made  on  the  same  day  (within-group  variance)  and  on  errors 
between  measurements  at  longer  intervals  (between-group  variance).  Each  set  of  weights  consisted 
of  four  pairs  of  one-piece  stainless  steel  weights  of  100  g,  25  g,  10  g,  and  1  g.  Data  include  weighing 
errors  of  the  various  laboratories  and  Youden  plots  of  the  measurements.  Some  errors  proved  to  be 
larger  than  had  been  expected.  The  results  illustrate  the  need  for  verifying  the  accuracy  of  measure- 
ment on  some  rational  basis. 

Key  words:  Between-group  variance;  mass  comparisons  of  1969;  systematic  and  random  weighing 
errors;  within-group  variance. 


This  round  robin  test  is  of  especial  interest  because 
it  includes  15  laboratories  whicb  do  not  participate  in 
the  NBS  Measurement  Analysis  Program  for  Mass 
Measurement.  Two  laboratories  which  do  participate 
in  the  NBS  MAP  program  are  also  included.  They  are 
the  IBM  Measurement  Standards  Laboratory  in  San 
Jose,  California,  and  Mettler  Instrument  Corporation 
in  Princeton,  New  Jersey.  There  is  very  little  in- 
formation available  about  the  capabilities  for  mass 
measurement  in  laboratories  outside  the  NBS  MAP 
program.  These  test  comparisons  provide  a  sample 
of  the  errors  which  occur  in  various  standard 
laboratories. 

Eighteen  laboratories  expressed  a  desire  to  take  part 
in  the  mass  comparison  tests.  One  laboratory  with- 
drew before  the  start,  leaving  17.  Measurements  were 
made  by  the  National  Bureau  of  Standards  Mass 
Laboratory  at  the  beginning  and  again  at  the  end,  and 
by  Mettler  Instrument  Corporation  near  the  middle  of 
each  schedule,  and  by  16  other  laboratories. 

The  test  packages  consist  of  pairs  of  weights  of 
100  g,  25  g,  10  g,  and  1  g.  Each  participant  received 
two  packages  at  times  separated  by  several  weeks  or 
months.  In  order  to  reduce  the  time  required,  the 
participants  were  divided  into  two  groups.  One  group 
included  the  Midwest  and  far-West  areas.  The  other 
included  the  East  and  South.  Four  packages  were 
prepared  and  circulated,  two  for  each  group.  All 
four  packages  were  mailed  from  the  National  Bureau 
of  Standards  on  December  2,  1968.  Package  A-l 
(weights  1,  2)   was  completed  in  September  1969, 


Package  A-2  (weights  3,  4)  in  October  1969,  and 
Packages  B-l  (weights  5,  6)  and  B-2  (weights  7,  8) 
were  completed  in  April  1969.  The  weights  are  of 
one-piece  stainless  steel,  which  provides  freedom  from 
certain  sources  of  variability  which  would  be  associ- 
ated with  screw-knob  type  of  construction,  and  with 
plated  or  lacquered  surface  finish.  We  arranged  to 
take  test  data  on  pairs  of  weights  because  of  the  ad- 
vantages of  analyzing  data  in  pairs.  Youden  plots  can 
show  deviations  within  pairs  taken  on  the  same  day  or 
within  pairs  taken  on  different  days.  The  NBS  values 
also  make  it  possible  to  plot  the  errors  with  reference 
to  a  "zero  error." 

Two  questions  are  of  especial  interest:  (1)  What 
are  the  errors  of  measurement?  and  (2)  How  well  is 
the  uncertainty  estimated?  Each  participant  was  in- 
structed to  weight  the  test  weights  as  they  would  weigh 
standard  weights.  The  method  to  be  used  was  left 
entirely  to  the  participant.  Each  participant  was  also 
asked  to  estimate  the  systematic  and  random  com- 
ponents of  the  uncertainty. 

The  results  of  the  test  are  shown  in  figures  1  through 
8  for  each  of  the  four  loads  in  the  two  packages. 
Weighing  errors  are  plotted  with  reference  to  the  NBS 
values  and  are  shown  as  circles  in  groups  of  four  for 
nominally  identical  weights.  Measurements  by  the 
National  Bureau  of  Standards  and  by  Mettler  are 
identified,  while  data  for  the  other  participants  is 
designated  by  code  number.  The  uncertainty  as  esti- 
mated by  the  participant  is  shown  by  a  vertical  line 
which  represents  the  plus  or  minus  sum  of  the  esti- 
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mates  for  the  systematic  and  random  components. 
The  precision  actually  obtained  is  shown  by  the 
scatter  within  the  groups  of  four.  In  64  percent  of 
the  groups,  the  precision  as  indicated  by  the  scatter  is 
comparable  to  the  scatter  of  the  NBS  measurements. 
By  this  I  mean  that  the  scatter  is  not  greater  than 
about  two  or  three  times  the  range  of  the  NBS  values. 

Youden  plots  of  the  data  in  figures  9  to  13  show  a 
pronounced  tendency  for  the  points  to  be  scattered 
along  a  diagonal  line,  indicating  the  presence  of  sys- 
tematic error.  In  these  figures  the  NBS  values  are 
shown  as  stars  and  the  group  mean  is  shown  as  an 
asterisk.  Comparison  of  figure  9,  showing  data  taken 
for  a  pair  on  the  same  day,  with  figure  13,  showing 
data  taken  for  a  pair  on  different  days,  indicates  that 
the  correlation  effect  is  slight  for  100  g  weights. 

To  summarize  these  results,  systematic  error  was 
larger  than  the  estimates  of  uncertainty  in  eight  of 
the  groups  of  four,  involving  five  different  laborato- 
ries. Imprecision  was  larger  than  anticipated  in  nine 
of  the  groups  of  four,  involving  six  different  labora- 
tories. It  appears  that  in  36  percent  of  the  groups  the 
precision  was  such  that  it  is  subject  to  improvement 
if  that  is  desired.  In  14  groups  of  four,  or  22  percent, 
the  estimate  of  uncertainty  was  too  small.    In  12 


groups  of  four,  or  19  percent,  the  estimate  of  uncer- 
tainty seems  unduly  large.  The  measurements  are 
much  better  than  their  estimates.  Thus  there  is  diffi- 
culty with  either  overestimation  or  underestimation  of 
the  uncertainty  43  percent  of  the  time. 

One  way  in  which  overestimation  of  the  uncertainty 
can  occur  is  by  reliance  on  manufacturer's  tolerances 
as  the  basis  for  accuracy  statements.  Any  tolerance 
statement  by  a  manufacturer  applies  to  the  limits  on 
the  error  in  a  large  number  of  items.  Most  of  the 
individual  items  will  have  errors  smaller  than  the 
limits  for  the  entire  group.  In  the  case  of  a  tolerance 
on  the  error  of  combinations  of  built-in  weights,  only 
certain  combinations  of  dial  settings  can  lead  to  the 
maximum  error.  The  remaining  combinations  will 
have  smaller  errors.  Tolerances  have  their  proper 
place  in  purchase  specifications,  but  more  detailed  in- 
formation is  needed  for  estimates  of  uncertainty. 

Some  simple,  rational  method  is  needed  in  order  to 
provide  a  basis  for  estimating  the  uncertainty.  The 
use  of  corrections  for  the  individual  weights  of  a  re- 
liable set  of  standard  weights  will  provide  more  pre- 
cise information  than  can  be  obtained  by  reference  to 
the  tolerances. 


Figure  1.  Errors  in  weighing  100-g 
weights,  group  A. 
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Figure  2.  Errors  in  weighing 
100-g  weights,  group  B. 
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Figure  4.  Errors  in  weighing 
25-g  weights,  group  B. 
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Figure  3.  Errors  in  weighing 
25-g  weights,  group  A. 
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Figure  5.  Errors  in  weighing  10-g  weights,  group  A. 
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Figure  6.  Errors  in 
weighing  10-g  weights, 
group  B. 
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Figure  8.  Errors  in  weighing 
1-g  weights,  group  B. 
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Ficure  7.  Errors  in  weighing  1-g  weights,  group  A. 
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Figure  9.  Youden  plot  of  100 -g 
Values,  package  A-l. 
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Figure  12.  Youden  plot  of  1-g  values,  package  A-l. 


Figure  10.  Youden  plot  of  25 -g 
values,  package  B-2. 
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Figure  13.  Youden  plot  of  100-g  data  taken  on  differ- 
ent days,  packages  A—l  and  A— 2. 
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NCSL  SPECIFICATION  PRESENTATION 

Frank  J.  Dyce 

Martin  Marietta  Company,  Orlando,  Fla.  32805 

This  paper  describes  in  summary  the  requirements  of  the  National  Conference  of  Standards 
Laboratories  "Calibration  System  Specification."  It  indicates  how  the  specification  was  written  by 
members  of  the  NCSL  Specifications  Committee  in  answer  to  a  DOD  request.  The  specification 
contains  the  minimum  essential  requirements  of  a  calibration  system.  The  paper  discusses  the 
steps  required  to  approve  the  specification  as  a  recommended  practice  and  to  obtain  Government 
approval  and  adoption. 

Key  words:  Calibration  records;  calibration  system  specification;  intervals;  procedures;  recall; 
reliability ;  traceability. 


1.  Introduction 

In  May  of  1969,  our  committee  published  a  compila- 
tion of  ten  government  specifications  which  affect  stand- 
ards and  calibration  laboratories.  A  paper  describing 
the  compilation  was  presented  at  the  1968  Standards 
Laboratories'  Conference  at  Boulder,  Colorado;  at 
that  time  several  representatives  from  DOD  suggested 
that  NCSL  should  write  its  own  specification.  This 
request  was  followed  up  by  a  questionnaire  which  in- 
dicated unanimous  support  for  the  idea.  The  ques- 
tionnaire results  also  revealed  19  specifications  which 
were  omitted  from  the  original  compilation,  all  of 
which  had  calibration  system  controls  in  them.  The 
committee  felt  that  our  profession  was  not  so  com- 
plicated that  we  required  29  specifications  to  control 
ourselves.  In  addition,  the  specifications  were,  in 
some  cases,  conflicting  and  in  many  cases,  ambiguous; 
therefore  we  have  prepared  a  NCSL  calibration  system 
specification  delineating  the  minimum  essential  re- 
quirements of  a  calibration  system  for  control  of 
measuring  instruments  and  standards.  This  paper  will 
describe  in  some  detail  that  specification. 

2.  The  Index 

The  first  step  in  preparation  of  the  specification  was 
the  decision  as  to  what  it  should  contain.  The  major 
categories  discussed  in  the  compilation  were  selected 
and  with  some  combination  and  rearrangement,  yielded 
the  following  index: 

INDEX 

1.  Purpose 

2.  Government  Audit 

3.  Subcontractor  Controls 

4.  System  Description 


5.  Records 

6.  Recall  System 

7.  Calibration  Intervals 

8.  Labeling 

9.  Calibration  Procedures 

10.  Reliability 

11.  Traceability  to  National  Standards 

12.  Environmental  Controls 

The  task  of  preparing  these  sections  was  assigned  to 
individual  committee  members.  The  results  were  com- 
piled and  circulated  for  comments  among  the  entire 
committee.  The  following  is  a  summary  of  the  re- 
quirements specified.  Reference  is  made  to  the  authors 
of  the  original  draft  of  each  section. 

1.  Purpose  Author:  Frank  J.  Dyce 
This  specification  defines  the  minimum  essential 
requirements  of  a  calibration  system  for  control 
of  measuring  instruments  and  standards.  It  ap- 
plies to  those  laboratories  which  specify  it  as 
their  governing  document. 

2.  Government  Audit  Author:  Frank  J.  Dyce 
Calibration  system  audits  should  be  restricted  to 
those  items  specifically  covered  by  the  applicable 
contract  requirements. 

3.  Subcontractor  Controls      Author:  Frank  J.  Dyce 

The  prime  contractor  shall  be  responsible  for  his 
subcontractors  having  a  calibration  system  that 
meets  the  requirements  of  this  specification  as  it 
applies  to  the  products  furnished. 
(Since  this  specification  covers  the  minimum  es- 
sential requirements,  it  is  the  contractor's  respon- 
sibility to  insure  that  the  subcontractor  meets  at 
least  these  requirements.) 
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4.  System  Description  Author:  John  Skinner 

A  written  description  of  the  calibration  system 
shall  be  maintained.  It  may  be  a  single  document 
or  several  documents  which  define  how  the  re- 
quirements of  this  specification  are  met.  It  shall 
include  at  least  the  following  items: 
Organization:  A  basic  statement  which  defines  the 
responsibility  and  authority  of  the  calibration 
laboratory. 

Measurement  Equipment  Controls:  Selection  of 
initial  intervals,  recall  procedures,  records,  labels. 
Technical  Control  System:  Outside  sources,  en- 
vironment, interval  adjustment. 
Calibration  Capability  List 

Terminology :  Definitions  of  special  terms  having 
significant  usage  in  the  calibration  system. 
(This  section  states  specifically  that  the  system 
description  will  contain  only  those  items  necessary 
for  a  procedural  review  of  a  calibration  program. 
It  permits  the  use  of  several  documents  which 
may  be  standard  company  procedures.  I  have 
personally  seen  a  200-page  book  put  together  to 
meet  previous  system  description  requirements.) 

5.  Records  Author :  James  Gilbert 

Records  shall  be  maintained  for  each  item  of 
measuring  equipment  upon  which  a  calibration 
has  been  performed. 

Records  shall  be  maintained  for  a  minimum  of 
one  year  plus  one  calibration  interval. 
Equipment  Identification  Records:  A  nonrepeti- 
tive  identification  shall  be  assigned  to  each  item 
of  measuring  equipment  in  the  calibration  system. 
Equipment  Location  Records:  Records  shall  be 
maintained  to  provide  information  necessary  to 
locate  equipment  due  for  calibration  or  repair. 
Calibration  History  Records:  Records  shall  be 
maintained  which  indicate  the  present  calibration 
status  and  previous  calibration  history  of  measur- 
ing equipment. 

Calibration  Control  Records:  Records  shall  be 
maintained  on  all  measurement  equipment  requir- 
ing calibration  to  provide  the  information  neces- 
sary for  the  control  and  audit  of  the  calibration 

system. 

I  This  section  limits  records  to  calibrated  equip- 
ment and  limits  the  length  of  time  required  for 
keeping  the  records.  It  provides  flexibility  in 
identifying  and  locating  equipment  and  lists  the 
minimum  specific  requirements  of  the  record  sys- 
tem.) 

6.  Recall  System  Author :  William  McCallum 

A  documented  system  shall  be  established  with  the 
purpose  that  no  standards  or  measuring  equip- 
ment with  an  expired  calibration  interval  shall  be 
used  for  a  product  measurement. 
User  Notification:  The  equipment  user  should  be 
notified  on  or  before  the  calibration  due  date. 
Equipment  Availability:  Items  must  be  made  avail- 
able for  calibration  on  or  before  their  calibration 
due  date  or  be  removed  from  service,  conspicu- 
ously tagged,  or  restricted  from  use. 


(This  section  clearly  states  the  purpose  and  re- 
quirements of  a  mandatory  recall  system.  It 
leaves  no  room  for  misinterpretation.) 

7.  Calibration  Intervals  Author:  Don  Greb 

Measuring  equipment  and  standards  shall  be  cali- 
brated at  periodic  intervals  designed  to  assure  that 
specified  accuracy  shall  be  maintained. 
Interval  Application:  Intervals  may  be  applied  to 
single  items  or  groups  of  items  which  are  rea- 
sonably homogeneous  with  respect  to  family  and 
reliability. 

Interval  Assignment:  Calibration  intervals  shall 
be  assigned  for  the  maximum  probability  that  the 
item(s)  will  be  within  accuracy  specification  when 
returned  for  calibration.  This  probability  will  be 
consistent  with  calibration  system  organizational 
requirements  and  resources.  Reliability,  failure, 
usage  and  cost  should  be  factors  considered  in  as- 
signing calibration  intervals. 

Interval  Adjustment:  Calibration  intervals  shall 
be  adjusted  as  necessary  on  the  basis  of  calibra- 
tion data  or  other  information  which  may  support 
a  change. 

Interval  Shortening:  Calibration  intervals  shall  be 
shortened  on  time  or  usage  dependent  instruments 
to  assure  continued  accuracy. 
Interval  Lengthening:  Intervals  may  be  lengthened 
when  indications  are  that  it  will  not  reduce  re- 
liability below  desired  minimums. 
Interval  Extensions:  Intervals  may  be  extended  by 
laboratory  supervision  due  to  critical  test  or  pro- 
duction schedules  or  other  requirements  provided. 
Historical  data  support  the  extension  and/or 
product  acceptance  is  withheld  pending  the  re- 
sults of  the  eventual  calibration. 
(This  section  discusses  assignment,  adjustment 
and  extension  of  intervals  in  a  complete  unambig- 
uous format.  It  permits  both  fixed  and  variable 
intervals,  and  extensions  within  limitations.) 

8.  Labeling  Author :  Frank  J.  Dyce 

All  measurement  equipment  and  standards  shall 
be  labeled  or  coded  to  indicate  their  calibration 
status.  Labels  shall  include  date  calibrated,  re- 
calibration  due  date,  identification  of  calibration 
personnel,  equipment  identification,  calibration 
agency. 

Label  Impractical:  When  labels  are  impractical, 
codes  should  be  used  or  recall  records  should  be 
monitored. 

Nonstandard  Labels:  "Calibration  Deviation," 
"Not  Required,"  "Do  Not  Use"  and  outside  agen- 
cy labels  should  be  used  as  applicable  to  cover 
deviations  from  normal  calibration  technique. 

9.  Calibration  Procedures         Author:  R.  R.  Willett 

A  current  document  shall  exist  containing  suffi- 
cient information  for  the  calibration  of  measuring 
instruments  and  standards  to  the  required  accu- 
racy for  product  measurements. 
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Contents:  It  may  be  a  local  document,  another 
company's  procedure,  a  manufacturer's  procedure 
or  a  composite  of  the  three.  It  should  be  refer- 
enced under  one  cover  with  local  approval. 
Format:  The  local  procedures  shall  include  the 
following  information — application,  specification, 
preliminary  operations,  step-by-step  procedure; 
calibration  table,  worksheet  and  appendix. 
Usage:  Locally  approved  calibration  procedures 
shall  be  followed  in  the  calibration  of  measuring 
equipment  and  standards  and  shall  be  available 
in  the  calibration  area. 

(This  section  basically  combines  the  requirements 
from  several  specifications.  It  has  sufficient  flexi- 
bility to  permit  use  of  existing  legitimate  proce- 
dures, yet  enough  firmness  to  require  complete 
procedures.) 

Reliability  Author:  M.  H.  Brenner 

The  calibration  program  shall  be  conducted  to 
maintain  measurement  equipment  above  a  speci- 
fied minimum  acceptable  reliability  level  such  that 
instruments  remain  within  their  certified  specifi- 
cation limits  over  their  calibration  interval. 
Minimum  Reliability  Level:  It  shall  be  the  ob- 
jective of  the  calibration  program  to  achieve  and 
maintain  a  minimum  documented  reliability  level 
consistent  with  organizational  requirements  and 
resources. 

Family  Reliability:  An  analysis  on  a  manufac- 
turer model  number  basis  shall  yield  overall  re- 
liability numbers  consistent  with  this  goal. 
Corrective  Action:  The  individual  instrument  or 
family  intervals  shall  be  adjusted  or  the  instru- 
ment specifications  deviated  (where  usage  require- 
ments permit)  as  required  to  obtain  the  relia- 
bility goal. 

Certified  Specification  Limits:  Upon  receipt  for 
calibration,  the  instrument  must  be  tested  to  de- 
termine if  it  is  within  tolerance  prior  to  any  ad- 
justment or  repair. 

Reliability  Records:  Shall  be  monitored  period- 
ically for  number  of  instruments,  number  of  cali- 
brations and  reliability  by  instrument  family. 
Reliability  Below  Minimum:  If  the  family  relia- 
bility drops  below  the  minimum  level,  one  or  more 
of  the  following  should  occur:  interval  adjust- 
ment, specification  deviation,  failure  analyses, 
calibration  procedure  change,  or  application  in- 
vestigation. 

Reliability  Above  Minimum:  If  the  reliability  re- 
mains consistently  above  the  minimum  level,  the 
calibration  interval  should  be  increased  to  the 
point  where  the  minimum  level  can  be  maintained 
with  optimum  equipment  utilization  and  minimum 
service  costs.  (This  is  a  requirement  which  is 
only  briefly  covered  in  most  specifications;  how- 
ever, it  is  an  area  we  felt  needed  strong  coverage. 
Reliability  emphasis  separates  the  good  from  the 
bad  programs.  The  actual  minimum  reliability 
level  will  vary  between  companies  but  it  should 
be  documented.) 


11.  Traceability  to  National  Standards 

Author:  Carl  Boyer 

The  calibration  program  shall  provide  the  means 
to  trace  the  accuracy  of  all  calibrated  measure- 
ment equipment  through  all  echelons  of  standards 
to  a  value  in  terms  of  U.  S.  National  Standards. 
Documented  Traceability:  Where  applicable,  cali- 
bration of  measurement  equipment  shall  have  doc- 
umented traceability  except  for  Ratio  Standards, 
Natural  Physical  Constants,  and  Derived  Stand- 
ards. 

Traceability  Documentation  :  Documentation  which 
substantiates  and  identifies  the  means  of  achieving 
traceability  shall  include  as  necessary:  calibration 
agency,  standards  used,  calibration  technique, 
data,  uncertainty,  environmental  conditions,  date 
of  calibration,  and  identification  of  the  calibra- 
tion personnel. 

12.  Environmental  Conditions  Author:  Frank  J.  Dyce 

The  calibration  environment  should  be  controlled 
to  the  extent  required  by  the  most  environmentally 
sensitive  measurement  made  in  the  area. 
Compensating  Corrections :  Where  control  is  in- 
adequate, compensating  corrections  should  be 
added  to  the  calibration  results. 
Environmental  Records:  Records  of  environmental 
conditions  are  only  necessary  when  the  require- 
ments are  tighter  than  the  laboratory  norm  or  the 
measurement  is  based  solely  on  the  environment 
it  was  calibrated  within. 

Environmental  Considerations :  Particular  atten- 
tion should  be  given  to  the  following  environmen- 
tal conditions  in  developing  calibration  techniques 
and  evaluating  measurement  data: 
Temperature,  voltage,  humidity,  dust,  vibration, 
air  currents,  RFI,  E.M.I.,  line  voltage,  grounding, 
and  lighting. 

(Environment  has  to  be  only  as  good  as  required 
by  the  measurements.  Recording  is  performed 
only  when  necessary  to  interpret  results  and  cor- 
rections are  applied  only  when  required.) 

3.  Conclusion 

This  specification  delineates  the  minimum  essential 
requirements  of  an  equipment  calibration  system  in  an 
unambiguous  format.  We  will  have  a  committee  meet- 
ing during  this  conference  to  make  final  changes  to  it. 
It  will  then  be  submitted  to  the  NCSL  Board  of  Di- 
rectors for  preliminary  approval  before  presentation 
to  DOD  for  their  comments.  The  minimum  we  hope 
to  achieve  with  this  document  is  NCSL-recommended 
practice  which  will  be  adopted  by  laboratories  not 
presently  working  to  a  Government  specification.  Sev- 
eral of  our  committee  members  indicated  interest  along 
these  lines. 

The  maximum  we  hope  to  achieve  with  this  docu- 
ment is  to  obtain  DOD  approval  and  have  them  specify 
it  in  lieu  of  the  other  29  specifications. 

We  feel  that  with  this  document  we  have  established 
a  platform  from  which  DOD  can  view  what  industry 
feels  are  the  requirements  of  a  calibration  system.  We 
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have  also  given  the  new  laboratories  a  general  outline 
of  what  they  must  do  to  develop  a  calibration  control 
system.  The  various  sections  of  this  document  can 
be  expanded  into  a  recommended  practice  just  as  has 


been  done  with  the  recommended  practices  on  cali- 
bration intervals  and  calibration  procedures.  We  will 
then  have  a  detailed  document,  corresponding  to  this 
general  one,  available  to  our  membership. 
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NCSL  70 


COST  VISIBILITY  EXCHANGE  PROGRAM— 
A  NEW  APPROACH  TO  COOPERATIVE  SAVINGS 

R.  J.  Barra 

Westinghouse  Defense  &  Space  Center,  Baltimore,  Md.  21203 

A  preliminary  survey  by  the  Calibration  Systems  Management  Committee  indicates  600,000 
manhours  per  year  spent  on  9  types  of  instruments,  including  oscilloscopes.  Member  organizations 
are  asked  to  contribute  cost  data  with  the  objective  of  saving  one  million  dollars  in  calibration  and 
maintenance  costs  by  1972. 

Key  words:  Calibration  and  maintenance  manhours;  digital  counters;  digital  voltmeters;  exchange 
program  on  costs;  oscilloscopes. 


a  new  cost  visibility  exchange  program: 


OBJECTIVE  . 


The  1969  NCSL  Directory  listings  show  about  5,000 
persons  working  primarily  in  standards  laboratories. 
Assuming  that  a  similar  number  of  people  are  work- 
ing in  lower  echelon  laboratories,  the  total  population 
becomes  10,000.  At  2,000  manhours  per  man  year, 
the  total  expenditure  is  twenty  million  manhours  per 
year.  As  you  can  see,  hundreds  of  millions  of  dollars 
are  involved. 

Why  so  much?  What  are  we  spending  these  man- 
hours  on?  What  instruments?  If  we  knew  what  the 
dollars  were  being  spent  on,  we  might  be  able  to  do 
something  about  it.  The  Calibration  Systems  Man- 
agement Committee  of  NCSL  had  to  know  the  answers 
to  these  questions.  To  be  most  effective  we  have  to 
concentrate  on  the  high-cost  areas.  What  we  wanted 
was  overall  cost  visibility.  But  the  committee  mem- 
bers alone  should  not  be  the  only  ones  to  benefit  from 
this  greater  cost  visibility.  Each  laboratory  manager 
would  like  to  know  how  his  costs  compare  with  costs 
in  other  laboratories.  With  the  capability  to  make 
cost  comparisons,  we  could  detect  otherwise  hidden 
areas  where  cost  reductions  might  be  achieved.  The 
exchange  of  cost  data  would  not  only  provide  broader 
visibility  to  all  laboratory  managers,  but  it  would  also 
provide  first,  a  communication  link  between  labs  with 
common  interests,  and  second,  a  foundation  for  coop- 
erative action  on  improved  methods  and  procedures 
leading  to  substantial  cost  reductions  for  all  participat- 
ing laboratories. 

Recognizing  ( 1 )  that  hundreds  of  millions  of  dol- 
lars were  involved,  (2)  that  present  cost  visibility  is 
quite  limited,  and  (3)  that  cooperative  action  was  al- 
most non-existent,  the  committee  established  the  fol- 
lowing objectives: 

•  To  develop  the  management  control  techniques  re- 
quired for  optimum  performance  of  calibration  sys- 
tems. 


melt  the  ice 

Figure  1.  A  new  cost  visibility  exchange  program: 
objective  .  .  .  melt  the  ice. 

•  To  provide  cost  visibility  to  managers. 

•  To  provide  means  to  achieve  cost  reductions,  with- 
out degradation  of  quality,  through  improved  meth- 
ods and  procedures. 
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Figure  2.  Committee  organization  chart. 


DIGITAL  VOLTMETERS 
J.  L.  Rickert 


Recognizing  that  the  committee  would  be  more  ef- 
fective if  its  members  had  clear  and  measurable  goals, 
the  following  specific  goals  were  established: 

1.  To  collect  2  million  manhours  of  specific  cost 
data. 

2.  To  disseminate  these  data  to  participating  labora- 
tories in  the  form  of  cost  visibility  reports. 

3.  To  provide  the  means  for  NCSL  members  to  save 
a  million  dollars  before  the  1973  NCSL  Conference. 

The  committee's  plan  of  action  is  as  follows: 
Phase  1 — Determine  where  the  calibration  dollars  are 
going.  Conduct  a  survey  to  find  those  items 
which  have  the  greatest  potential  cost  reduc- 
tion to  the  greatest  number  of  NCSL  mem- 
bers. 

Phase  2 — Collect,  analyze  and  report  the  cost  data  on 

these  groups  of  instruments. 
Phase  3 — Restructure  the  committee  into  sub-commit- 
tees for  in-depth  probings. 
Phase  4 — Collect  and  develop  cost  reduction  ideas,  and 
provide  a  means  to  exchange  them. 
The  committee's  progress  toward  the  attainment  of 
our  goals  can  best  be  measured  by  reviewing  actions 
to  date.  Less  than  a  year  ago  Phase  1  began.  On 
July  30,  1969,  a  questionnaire  on  calibration  interval 
versus  cost  was  sent  to  all  delegates  and  sponsors. 
About  25  percent  of  the  membership  responded.  The 
data  collected  exceeded  one  million  calibration  and 
maintenance  hours.  Sixty  percent  of  these  hours  were 
associated  with  just  nine  groups  of  instruments.  The 
committee  achieved  one  of  its  first  objectives;  that  is, 
we  found  out  where  most  of  our  calibration  dollars 
were  going.  The  committee's  first  cost  visibility  re- 
port, covering  these  nine  groups  of  instruments,  is 
given  in  the  appendix.  How  does  your  lab  compare 
with  the  data  presented?  Are  your  intervals  longer 
or  shorter?  Are  your  annual  calibration  maintenance 
hours  lower  or  higher? 
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Figure  3.  Results  of  calibration  cost  survey, 
Fall  1969:  Percent  total  hours. 


As  listed  in 
ments  are: 


the  report,  the  nine  groups  of  instru- 

Oscilloscopes 
Oscillators/Generators 
Electronic  Meters 
Oscilloscope  Plug-ins 
Electronic  Counters 
General  Lab  Meters 
Differential  Voltmeters 
Pressure  Gages 
Digital  Voltmeters 
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How  large  a  sample  did  these  data  represent?  Not 
too  large.  For  example,  a  few  comparative  figures  are 
given  below: 

A.  Oscilloscopes— 28,000  out  of  about  350,000 

B.  Counters  (Digital)— 6,700  out  of  175,000 

C.  Digital  Voltmeters— 1,500  out  of  75,000 
Using  the  weighted  average  figures  from  the  survey, 
the  total  calibration  and  maintenance  effort  for  these 
instruments  is  over  3,300,000  manhours  per  year.  The 
breakdown  is  as  follows: 

A.  Oscilloscopes — 350,000  at  5.8  manhours/year/ 

scope  equals  2  million  man- 
hours. 

B.  Counters — 175,000  at  3.9  manhours/year/ 

counter  equals  700,000  manhours. 

C.  Digital  Voltmeters— 75,000  at  7.8  manhours/ 

year/ DVM  equals  600,000 
manhours. 

As  you  can  readily  see,  the  cost  reduction  potential  is 
significant.  Assuming  that  the  above  figures  are  rep- 
resentative of  the  total  population,  the  one  million 
manhours  of  the  survey  is  only  about  5  percent  of  the 
total  effort,  which  figures  to  be  about  20  million  man- 
hours. 

This  first  cost  visibility  report  was  just  part  of 
Phase  2.  This  month  you  will  have  a  second  chance 
to  be  a  contributor  to  this  program.  The  committee 
needs  additional  data  to  supplement  the  information 
received  from  the  1969  survey.  Cost  visibility  ex- 
change cards  are  being  sent  to  all  delegates.  We  are 
looking  for  specific  cost  information  on  these  groups 
of  instruments.  Send  in  as  many  cards  as  you  wish. 
If  you  are  interested  in  getting  data  on  all  these  groups 
of  instruments,  then  a  card  for  each  group  must  be 
contributed.  Each  card  is  your  ticket  to  greater  cost 
visibility  and  an  opportunity  to  achieve  substantial 
cost  reductions  for  your  laboratory. 


QUANTITY 

HRS/YR 

AVG  HRS/YR/ITEM 

Oscilloscopes 

28,373 

169.912 

5.8 

Oscillators,  Signal  Generators 
&  Function  Generators 

30.813 

146.082 

5.4 

Electronic  Meters  (V.T.V.M.'s  & 
Electronic  Multimeters) 

44,663 

102.606 

2.1 

Scope  Plug- Ins 

17,073 

47.870 

2.2 

Counters 

6.741 

36.642 

3,9 

General  Lab  Meters  (Panel  Meters, 
Portables  &  V.O.M'sl 

15,989 

35.089 

1.9 

Differential  Voltmeters 

7.209 

29.405 

3.3 

Pressure  Gages 

9.580 

26.265 

1.9 

Digital  Voltmeters 

1,540 

14.513 

7.8 

Totals 

161,981 

608.384 

Table  1.  Instruments  with  highest  calibration  I 
maintenance  hours 


CAL/MAINT 
HRS/INST/YR 

CALIBRATION  INTERVALS  (MONTHS) 

TOTALS 

2 

3 

4 

6 

9 

12 

2.2-5 

76 

11.504 

580 

3641 

47 

125 

67 

15.259 
781 

6  8 

159 

759 

6180 

2428 

25 

523 

65 

64 

6269 

3934 

9-12 

653 

312 

42 

43 

380 

34 

76 

76 
1464 

MORE  THAN  12 

33 
249 

50 
115 

134 

1 

83 
499 

TOTAL  INST 

812 

1429 

20.942 

4711 

271 

208 

28.373 

TOTAL  HRS 

8253 

15,095 

117.803 

25.688 

1695 

1378 

169.912 

AVG  HRS/YR/INST 

10.2 

10.6 

5.6 

5.5 

6.3 

6.6 

5.8' 

GOVERNMENT 

INDUSTRY 


•WEIGHTED  AVERAGE 


Table  2.  Oscilloscopes 


CAL/MAINT 

CALIBRATION  INTERVALS  (MONTHS) 

TOTALS 

HRS/INST/YR 

2 

3 

4 

5 

6 

7 

9 

12 

2  4 

127 

50 

12.614 

2302 

1014 

730 
154 

147 

16,357 

331 

4-8 

490 

2416 

90 

4628 

6 

3972 
918 

193 

11,209 
1504 

9-12 

10 

184 

115 

309 

MORE  THAN  12 

488 

52 

505 

57 

1 

505 

598 

TOTAL  ITEMS 

488 

679 

3302 

16,798 

7308 

1014 

884 

340 

30,813 

TOTALS  HRS 

10.367 

5227 

26.428 

64.066 

31,967 

3486 

2833 

1708 

146,082 

AVG  HRS/INST/YR 

21.2 

7.7 

8.0 

3.8 

4.4 

3.4 

3.2 

5.0 

5.4* 

GOVERNMENT 

INDUSTRY 


•WEIGHTED  AVERAGE 


Table  3.  Oscillators,  signal  and  function  generators 
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CAL/MAINT 
HRS/INST/YR 

CALIBRATION  INTERVALS  (MONTHS) 

TOTALS 


1 

2 

3 

4 

5 

6 

9 

12 

1.5-2.5 



— 

— 

— 

13,398 

5534 

21,674 

— 

40,606 

46 

66 

272 

■MA 

3  6 

— 

— 

— 

55 

47 

66 

— 

349 

547 

150 

613 

33 

72 

806 

127 

90 

1891 

7  -  10 

— 

— 

— 

15 

— 

— 

— 

— 

15 

90 

6 

27 

416 

33 

72 

9 

653 

11  100 

12 

12 

227 

76 

46 

6 

— 

— 

— 

— 

355 

TOTAL  ITEMS 

317 

232 

732 

603 

13,650 

6478 

22.182 

469 

44,663 

TOTAL  HRS 

4221 

1786 

4043 

3759 

30,922 

15,850 

40.368 

1657 

102,606 

AVG  HR/INST/YR 

13.0 

7.7 

5.5 

6.2 

2.3 

2.4 

1.8 

3.5 

2.1* 

GOVERNMENT 

INDUSTRY 


♦WEIGHTED  AVERAGE 


Table  4.  Electronic  meters  {pointer  type,  active  circuit) 


CAL/MAINT 

CALIBRATION  INTERVALS  (MONTHS) 

TOTALS 

HRS/INST/YR 

2 

3 

4 

5 

6 

12 

1.3-2.5 

11,364 

114 

11.364 
114 

2  6  5 

1254 

65 

90 

2420 

604 

210 

2420 

2223 

6  10 

339 

184 

36 

1 

559 

11  40 

229 

2 

18 

5 

113 

25 

131 

262 

TOTAL  ITEMS 

568 

1440 

124 

11.454 

3277 

210 

17.072 

TOTAL HRS 

7870 

5098 

974 

20.773 

12.578 

604 

47.870 

AVG  HRS/INST/YR 

13.4 

3.5 

7.6 

1.8 

38 

2.8 

2.2* 

GOVERNMENT 

INDUSTRY 


•WEIGHTED  AVERAGE 


Table  5.  Scope  plug-ins 


CAL/MAINT 
HRS/INST/YR 

CALIBRATION  INTERVALS  (MONTHS! 

TOTALS 

30R  LESS 

4 

5 

6 

9 

12 

0  8  0.9 

1048 

1048 

1  ■  2 

2576 

6456 

1452 

10.484 

324 

85 

409 

2.1  4 

1794 

562 

2356 

300 

406 

570 

1 

1277 

5  -  35 

180 

180 

232 

2 

1 

235 

TOTAL  INST 

532 

2576 

1794 

8236 

2202 

649 

15.989 

TOTAL  HRS 

3201 

5558 

5918 

12.194 

6618 

1600 

35.089 

AVG  HRS/INST/YR 

60 

2.2 

3.3 

1.5 

3.0 

2.5 

1.9* 

GOVERNMENT 
INDUSTRY 


•WEIGHTED  AVERAGE 


Table  7.  General  lab  meters  (pointer  type, 
passive  circuit) 


CAL/MAINT 

CALIBRATION  INTERVALS  (MONTHS) 

TOTALS 

HRS/INST/YR 

30R  LESS 

4 

5 

6 

12 

1.5-2 

1310 

1310 

3  5 

6 

2490 

1 

2490 

7 

6  10 

117 

850 

1274 

13 

84 

1 

934 

14C5 

11  20 

281 

169 

75 

525 

21  60 

5 

57 

8 

5 
65 

TOTAL  INST 

466 

4791 

1310 

89 

85 

6741 

TOTAL  HRS 

7068 

25.719 

2210 

1182 

463 

36,642 

AVG  HRS/INST/YR 

15.2 

5.4 

1.7 

13.3 

5.4 

3.9* 

GOVERNMENT 

INDUSTRY 


•WEIGHTED  AVERAGE 


Table  6.  Counters 
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CAL/MAINT 
HRS/IIMST/YR 

CALIBRATION  INTERVALS  (MONTHS) 

TOTALS 

1 

2 

3 

4 

6 

2.4  -  3.0 

3940 

3940 

- 

1314 

- 

3.1  -4 

— 
— 

— 
— 

1310 
— 

— 
— 

— 

1 

1310 

1 

5  ■  10 

— 

297 

— 

88 

385 

11  -  50 





12 

- 

12 

147 

33 

53 

14 

247 

TOTAL  INST 

147 

33 

5600 

26 

89 

7209 

TOTAL  HRS 

5662 

4120 

18.735 

356 

532 

29,405 

AVG  HRS/INST/YR 

38.5 

3.1 

3.3 

13.7 

6.0 

3.3* 

GOVERNMENT  'WEIGHTED  AVERAGE 

INDUSTRY 


Table  8.  Differential  voltmeters 


CAL/MAINT 
HRS/INST/YR 

CALIBRATION  INTERVAL  (MONTHS) 

TOTALS 

3 

4 

6 

12 

0.9-1.0 

164 

164 

2364 

2364 

2-3 

3238 

3238 

396 

57 

1000 

1453 

4-5 

1615 

1615 

6  OR  MORE 

111 

630 

5 

746 

TOTAL  INST 

507 

57 

6483 

2533 

9580 

TOTAL  HRS 

2232 

179 

21.323 

2531 

26,265 

AVG  HRS/INST/YR 

4.4 

3.0 

3.3 

0.9 

1.9- 

GOVERNMENT  'WEIGHTED  AVERAGE 

INDUSTRY 


Table  9.  Pressure  gages 

Phase  3  has  also  been  implemented.  The  committee 
has  been  restructured  into  an  organization  with  eight 
sub-committees.  The  organization  chart  is  shown  in 
figure  2.  Several  delegates,  who  have  contributed  a 
large  portion  of  the  data  on  these  instruments,  have 
already  been  appointed  to  serve  on  subcommittees 
that  are  of  major  interest  to  them.  A  planning  and 
control  staff  has  also  been  established  to  assist  the 
committee  chairman  in  the  establishment  of  common 
guidelines  for  the  subcommittee  chairmen  to  follow. 
However,  each  subcommittee  chairman  will  be  given 
maximum  freedom  to  operate  his  subcommittee  auton- 


CAL/MAINT 
HRS/INST/YR 

CALIBRATION  INTERVALS  (MONTHS) 

TOTALS 

1 

2 

3 

4 

6 

4  -  5 





557 

131 

22 

710 

6  -  10 

190 

190 



141 

17 



158 

11-15 

44 

60 

129 





233 

16  ■  20 

81 

7 

5 

4 

93 

21  -35 

16 

16 

92 

14 

30 



136 

TOTAL  INST 

136 

312 

740 

136 

216 

1540 

TOTAL  HRS 

3119 

3945 

546S 

531 

1450 

14,513 

AVG  HRS/INST/YR 

229 

12.6 

7.4 

3.9 

6.7 

7.8" 

GOVERNMENT  'WEIGHTED  AVERAGE 

INDUSTRY 


Table  10.  Digital  voltmeters 

omously.  As  mentioned  previously,  the  key  objective 
of  each  subcommittee  will  be  to  provide  the  means 
for  members  to  achieve  cost  reductions. 

Phase  4  is  just  about  to  start.  Each  subcommittee 
chairman  will  be  responsible  for  providing  the  means 
for  NCSL  members  to  achieve  cost  reductions  asso- 
ciated with  his  instrument  group.  For  example,  some 
of  the  means  are: 

1.  Collection,  analysis,  and  reporting  of  cost  data; 

2.  Operation  of  an  information  exchange  pro- 
gram; 

3.  Operation  of  a  trouble-shooting  summary  li- 
brary; 

4.  A  telephone  directory  linking  delegates  with 
common  interests. 

Each  subcommittee  will  have  its  own  goals,  such  as: 

1.  Collection  of  a  specific  amount  of  data; 

2.  Reporting  of  this  data  on  schedule; 

3.  Helping  members  save  a  specific  amount  of 
dollars  in  a  specific  amount  of  time. 

When  all  the  subcommittee  goals  are  put  together 
they  will  meet  or  exceed  the  total  goals  of  the  com- 
mittee.   Once  again,  these  goals  are: 

1.  To  collect  2  million  manhours  of  cost  data. 

2.  To  help  NCSL  members  save  one  million  dollars 
before  the  1972  NCSL  conference. 

Be  a  CONTRIBUTOR. 
Help  MELT  THE  ICE. 
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SESSION  3 :  NEW  METHODS  OF  OPTIMIZING  INTERVALS 
Chairman:  Carl  Boyer 
Honeywell,  Inc.,  Annapolis,  M<1.  21404 

KEYS  TO  OPTIMUM  CALIBRATION  INTERVALS 
Rolf  B.  F.  Schumacher 

Autoneties  Division,  North  American  Rockwell,  Anaheim,  Calif.  92803 

The  search  for  optimum  calibration  intervals  or  for  methods  to  obtain  them  can  be  frustrating. 
Our  theories  concerning  the  behavior  of  measuring  instruments  are  deficient,  and  as  a  result,  so  are 
all  theoretical  models  attempting  to  establish  optimum  intervals.  Nor  can  we  hope  to  determine 
empirically  what  opimum  intervals  should  look  like.  The  many  variables  in  calibration  control 
systems,  affecting  quality  and  behavior  and  performance  evaluation  of  measuring  instruments, 
forbid  us  to  compare  the  meaning  and  merit  of  calibration  intervals  achieved  in  one  calibration 
control  system  with  those  achieved  in  another  such  system.  Time  does  not  permit  us  to  experiment 
in  one  system  until  we  achieve  calibration  intervals  which  are  even  near  optimum.  We  must  be  modest 
and  practical.   We  must  lower  our  sights. 

This  paper  attempts  to  show  the  main  sources  of  our  ignorance  about  instrument  behavior  and 
about  the  influence  of  a  calibration  control  system  on  instrument  behavior  and  on  our  evaluation  of 
this  behavior.  It  offers  seven  keys  to  the  establishment  of  an  efficient  method  for  calibration  interval 
adjustment.  Emphasis  is  on  efficiency,  practicability,  economy.  The  purpose  of  adjusting  calibration 
intervals  is  to  maintain  a  given  level  of  instrument  quality  level  at  the  lowest  cost.  Having  found  a 
satisfactory  calibration  interval  adjustment  method,  one  should  look  to  other  components  of  the 
entire  calibration  control  system  for  opportunities  to  lower  costs. 

Key  words:  Calibration  intervals;  economy  in  adjusting  quality  level;  measurement  uncertainty; 
reliability  of  working  instruments. 


1.  Introduction 

Calibration  intervals  and  the  problems  of  determin- 
ing them  have  been  with  most  of  us  in  industry  ever 
since  the  latter  half  of  the  1950's  when  the  Govern- 
ment began  requiring  periodic  recalibration  of  meas- 
uring tools  and  instruments  used  by  its  defense  con- 
tractors. The  principle  was  sound:  measuring  instru- 
ments deteriorate  with  time  and  use;  so,  once  in  a 
while,  they  have  to  be  recalibrated  to  make  sure  the 
measurements  made  with  them  are  correct.  Thus,  cali- 
bration intervals  were  introduced,  set,  and  adjusted,  to 
recall  measuring  instruments  periodically  to  insure  that 
each  is  within  its  tolerances  at  all  times  when  in  use. 

Soon,  however,  we  found  that  this  was  not  possible, 
and  we  had  to  be  satisfied  with  keeping  most  measur- 
ing instruments  in  tolerance  most  of  the  time.  But 
how  many,  and  how  much  of  the  time?  This  question 
brought  the  two  key  problems  of  interval  determina- 
tion into  clear  focus,  to  wit: 

1.  Finding  the  function  for  time  of  instrument  de- 
terioration (when  does  a  given  instrument  get  out  of 
tolerance? ) 

2.  Finding  the  optimum  quality  level  (how  many 


instruments  should  we,  and  can  we  afford  to,  have  in 
tolerance  at  any  one  time  when  in  use?)  [I]1 

Closely  connected  with  the  second  problem,  finding 
the  optimum  quality  level,  is  the  problem  of  measur- 
ing this  quality  level.  This,  in  turn,  leads  to  questions 
about  the  standards  against  which  the  quality  level  is 
measured,  the  tolerance  specifications  for  the  instru- 
ments, and  the  calibration  procedures — are  they  con- 
trolled, stable,  and  adequate? — about  policies,  person- 
nel, etc.  How  certain  are  we  that  an  instrument  stated 
to  be  in  tolerance  was  really  performing  the  way  we 
wanted?  Uncertainty  afflicts  the  measurement  of  the 
quality  level  of  our  instrumentation  and  blurs  our 
vision  of  instrument  deterioration.  Thus,  the  two  key 
problems  of  interval  adjustment,  namely  the  determin- 
ation of  instrument  quality  level  and  the  time-depend- 
ence of  instrument  deterioration,  are  connected  by  the 
same  factors  which  introduce  uncertainty  in  both. 

We  conclude  that  all  elements  of  a  calibration  con- 
trol system  are  inseparably  linked  together.  An  ex- 
amination of  calibration  intervals  leads  to  an  exam- 
ination of  the  entire  system.    In  the  following  pages 

1  Figures  in  brackets  indicate  the  literature  references  at  the  end  of 
this  paper. 
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we  first  examine  the  factors  influencing  calibration  in- 
tervals and  then  see  what  we  can  say  about  methods 
of  interval  determination  in  general  and  how  to  de- 
velop a  critique  for  them.  Unfortunately,  at  the  end 
of  this  critical  examination,  we  shall  not  find  a  method 
for  determining  calibration  intervals  that  can  be  gen- 
erally recommended.  But  we  shall  see  what  we  can 
do  to  select  a  calibration  interval  adjustment  method 
that  will  permit  optimizing  our  calibration  control 
system. 

To  do  that,  we  must  first  debunk  the  notion  that  any 
one  mathematical  approach  can  solve  the  problem  of 
obtaining  optimum  calibration  intervals.  We  must  be 
fully  aware  of  all  the  factors  influencing  calibration 
intervals  in  order  to  innovate  methods  which  will  yield 
optimum  intervals  and  more  cost-effective  calibration 
control  systems. 

This  paper  deals  only  with  the  problems  of  calibra- 
tion intervals  for  large  numbers  of  working  instru- 
ments. Where  calibration  intervals  are  determined  on 
the  basis  of  standards  observed  individually  and  ana- 
lyzed in  detail,  other  considerations  enter  and  have 
been  discussed  elsewhere  [2,  3] . 

2.  Factors  Determining  the  Length  of  a 
Calibration  Interval 

2.1.  Time  Dependence 

At  the  basis  of  the  concept  of  calibration  intervals 
is  the  notion  that  a  measuring  tool  or  instrument  de- 
teriorates as  the  effects  of  chemical  or  physical  action 
accumulate  with  time.  But  how  do  they  accumulate? 
Does  the  instrument  deteriorate  faster  when  we  use  it 
frequently?  Or  does  it  deteriorate  independently  of 
usage  ? 

In  most  cases  we  do  not  know.  There  will  be  some 
types  whose  deterioration  is  a  function  of  usage  alone. 
And  there  will  be  others  whose  deterioration  will  be  a 
function  of  their  exposure  to  the  atmosphere.  And 
there  will  be  still  others  whose  deterioration  is  a  func- 
tion of  something  else.  What  is  the  factor  contributing 
most  to  their  deterioration,  and  does  the  deterioration 
progress  linearly,  exponentially,  stepwise,  or  how? 
For  most  measuring  instruments  it  would  be  impossible 
to  find  the  answers  to  these  questions.  And  even  if 
we  could,  we  might  find  almost  as  many  patterns  of 
behavior  as  we  have  instrument  types.  Could  we  pos- 
sibly have  as  many  different  interval  adjustment  meth- 
ods?   Certainly  not. 

Our  ignorance  concerning  measuring  instrument  be- 
havior will  force  us  to  make  assumptions  when  estab- 
lishing calibration  intervals.  We  generally  make  those 
assumptions  which  appear  plausible  and  which  most 
likely  apply  to  at  least  some  of  our  instruments.  So, 
let  me  propose  that,  in  establishing  calibration  inter- 
vals, we  make  the  assumptions  that  help  us  develop  a 
method  that  is  inexpensive  to  implement  and  use. 

In  a  study  to  determine  whether  shorter  calibration 
intervals  produce  instrument  families  with  higher  in- 
tolerance ratios,  one  large  family  of  like  instruments 
by  one  manufacturer,  and  with  the  same  model  desig- 


nation, was  divided  into  four  groups  of  equal  size. 
Each  group  was  assigned  a  calibration  interval  of  5, 
10,  15,  or  20  weeks.  Figure  la  is  a  compilation  of 
the  in-tolerance  percentages  of  the  groups  measured  at 
recall.  ( Obviously  inoperative  instruments  were  ex- 
cluded from  the  count  and  charted  separately.)  The 
study  on  instrument  type  A  was  repeated  with  type  B 
(fig.  lb).  Were  the  calibration  intervals  a  major 
factor  determining  the  in-tolerance  percentages  of  the 
instruments  charted  in  figure  1?  There  is  no  clear 
answer,  but  the  tolerance  specifications  against  which 
the  instruments  are  calibrated  appear  to  be  a  more  im- 
portant factor  than  the  calibration  intervals  in  this 
limited  situation. 

One  additional  thought:  The  assumption  that  the 
moment  at  which  an  instrument  slips  out  of  tolerance 
is  uniformly  distributed  over  a  period  of  time,  rather 
than  as  some  function  of  time,  may  lead  us  to  some 
radically  new  methods  of  calibration  control.  How- 
ever, for  the  purpose  of  the  following  discussion,  I 
shall  assume  that  there  is  at  least  some  vague  rela- 
tionship between  an  instrument's  calibration  interval 
and  its  in-tolerance  condition,  and  that  a  low  in-tol- 
erance percentage  requires  short  calibration  intervals 
and  attendant  high  calibration  costs.  But  let  us  re- 
member that  these  statements  of  conventional  wisdom 
deserve  our  most  critical  questioning. 
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Figure  la.  Instrument  Type  A. 
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Figure  lb.  Instrument  Type  B. 

2.2.  Quality  Level 

By  instrument  quality  level  I  mean  the  number  of 
measuring  instruments  in  tolerance,  as  a  percentage 
of  all  instruments  under  consideration.  It  is  often  as- 
sumed that  this  level  is  linked  to  calibration  interval. 
Soon  after  it  was  realized  that  we  will  never  be  able 
to  have  all  measuring  instruments  in  tolerance  all  the 
time,  arguments  arose  about  how  many  we  must  have 
in  tolerance.  Some  people  maintain  that  95  percent 
of  all  instruments  should  be  in  tolerance  at  recall. 
Some  say  this  is  too  high  a  percentage;  others  main- 
tain it  is  too  low.  For  ail  practical  purposes,  we  do 
not  know  what  the  quality  level  should  be. 

We  should  stop  here  for  a  moment  and  consider  the 
predicament  we  are  in.  We  do  not  know  what  is  hap- 
pening to  measuring  instruments  (the  question  of  time 
dependence),  and  we  do  not  know  what  should  hap- 
pen (quality  level).  Are  our  calibration  interval  ad- 
justment methods  in  sad  shape?    You  bet  they  are! 

What  could  be  done  is  to  calculate  the  cost  of  a 
calibration  system  producing  a  given  quality  level,  say 
90  percent,  and  also  the  cost  of  an  alternate  system 
producing,  say  95  percent.  We  could  also  determine 
the  dollar  value  of  the  reduced  rework  and  scrap,  and 
estimate  the  incommensurate  effects,  like  increased 
customer  satisfaction.  If  there  are  no  incommensu- 
rates,  the  quality  level  we  want  is  the  one  which  mini- 


mizes the  difference  between  the  cost  of  obtaining  it 
and  the  dollar  value  of  the  benefits.  If  there  are 
incommensurates,  the  difference  is  the  cost  of  the  in- 
commensurate benefits.  Somebody  in  control  over  the 
system  can  then  decide  whether  the  benefits  are  worth 
the  costs  or  whether  he  would  like  to  buy  additional 
benefits  at  the  indicated  rate. 

Examples  of  how  an  optimum  quality  level  may  be 
determined  are  given  in  references  [1]  and  [6].  This 
brings  us  to  the  question  of  how  to  measure  this  qual- 
ity level. 

2.3.  Measuring  the  Quality  Level 

Like  all  measurements,  the  measurement  of  instru- 
ment quality  level  is  associated  with  uncertainties.  The 
sources  of  these  uncertainties  are  found  throughout 
the  entire  calibration  control  system.  One  large  and 
unnecessary  error  stems  from  the  practice  of  measur- 
ing quality  level  at  recall,  i.e.,  at  a  time  when  a  meas- 
uring instrument  is  back  in  the  calibration  laboratory 
for  recalibration.  What  we  really  want  to  know  is 
whether  the  instrument  was  in  tolerance  when  it  was 
being  used.  Therefore,  we  should  measure  this  quality 
level  in  use,  i.e.,  at  the  time  when  the  instruments  are 
being  used,  when  it  does  make  a  difference  whether 
an  instrument  is  in  tolerance  or  not.  The  effects  of 
measuring  at  recall  or  in  use  are  discussed  in  detail 
in  Appendix  A.l. 

Other  main  sources  of  uncertainties  which  prevent 
us  from  obtaining  the  instrument  quality  level  we  need 
are  found  in  the  tolerance  specifications  against  which 
instruments  are  calibrated,  and  in  the  choice  of  charac- 
teristics the  instrument  must  meet  (as  contrasted  with 
those  which  are  incidental ) .  Also,  the  procedures  we 
use  when  calibrating  an  instrument  influence  the  mag- 
nitude and  direction  of  systematic  errors  and  the  pos- 
sible magnitude  of  random  errors.  These  uncertainties 
becloud  our  decision  as  to  whether  an  instrument  is 
declared  in  tolerance  or  not,  and  whether  or  not  to 
change  calibration  intervals.  Then  there  are  person- 
nel policies,  instrument-handling  policies,  treatment  of 
tolerances,  etc.,  which  affect  that  decision.  For  a  more 
detailed  discussion  of  error  sources,  see  Appendix  A. 2. 

It  stands  to  reason  that  all  elements  which  are 
sources  of  uncertainty  should  be  examined  with  a  view 
to  reducing  the  chances  of  making  interval  changes  in- 
advisedly. But  more  than  that,  all  elements  must  be 
compatible  with  the  objectives  of  the  calibration  con- 
trol system.  For  instance,  instrument  tolerances  must 
be  only  as  tight  as  product  tolerances  call  for — not 
more,  not  less.  And  finally,  all  elements  must  be  com- 
patible with  each  other.  For  instance,  if  the  interval 
adjustment  method  requires  a  technician  to  know  the 
condition  of  an  instrument  when  it  was  recalibrated 
previously,  the  system  must  make  this  information 
available  to  him. 

Thus,  there  are  many  things  which  have  a  decisive 
influence  on  the  calibration  intervals  which  we  finally 
assign  to  measuring  instruments.  And  I  submit  that 
study  of  these  holds  more  promise  for  minimizing  cali- 
bration system  costs  and  optimizing  calibration  inter- 
vals than  a  concentration  on  the  mechanics  of  adjust- 
ing intervals. 
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3.  How  to  Determine  Optimum 
Calibration  Intervals 

As  explained  in  the  preceding  paragraphs,  we  are 
confronted  with  a  number  of  factors  which  we  suspect 
will  affect  the  quality  level  of  measuring  instruments. 
Since  calibration  intervals  are  a  function  of  this  qual- 
ity level,  they  are  also  a  function  of  all  these  factors 
and  their  unknown  effects. 

From  the  preceding  discussion,  we  can  derive  the 
following  seven  "Keys  to  Optimum  Calibration  Inter- 
vals": 

Key  No.  1 — There  is  no  optimum  calibration  interval 
per  se,  and  there  is  no  optimum  interval  adjustment 
method  per  se. 

Since  we  do  not  know  how  all  important  factors  of 
a  calibration  control  system  interact,  we  cannot  opti- 
mize calibration  intervals  or  interval  adjustment  meth- 
ods theoretically.  And  to  do  it  empirically,  by  trial 
and  error,  we  have  neither  the  time  nor  the  funds  in 
general.  Hence, 

Key  No.  2 — We  cannot  hope  to  optimize,  in  the  strict- 
est sense  of  the  word,  a  calibration  interval;  we  can 
only  hope  to  arrive  at  a  reasonably  satisfactory  one. 

"Optimum,"  i.e.  "fairly  cost-effective,  satisfactory" 
calibration  intervals  are  the  result  of  an  "optimization" 
of  all  important  factors  of  a  calibration  control  system, 
of  which  the  interval  adjustment  method  is  only  one. 

Key  No.  3 — An  interval  adjustment  method  can  be 
"optimized"  only  in  relation  to  a  specific  calibration 
control  system;  a  method  satisfactory  for  one  system 
may  be  unsatisfactory  for  another. 

Key  No.  4 — An  interval  adjustment  method  "optimiz- 
ing" intervals  for  a  given  control  system 

— is  economical  to  establish  and  to  operate, 
— causes  intervals  to  respond  promptly  to  changes 
in  the  system, 

— zeros  in  quickly  on  calibration  intervals  which 
optimize  the  quality-cost  relationship  of  the  system. 

Key  No.  4  states  the  main  criteria  for  selecting  a 
calibration  interval  adjustment  method. 

The  largely  unknown  factors  influencing  calibration 
intervals  cannot  be  described  by  a  mathematical  mod- 
ed;  assumptions  will  have  to  serve  in  lieu  of  missing 
knowledge.  Hence, 

Key  No.  5 — A  satisfactory  interval  adjustment  method 
is  one  which  yields  an  acceptable  quality  level  at  low 
cost,  regardless  of  the  underlying  assumptions,  as  long 
as  such  assumptions  do  not  contradict  our  experience. 
There  may  be  a  number  of  equally  satisfactory  meth- 
ods for  any  one  calibration  control  system. 
From  this  follows 

Key  No.  6 — An  established  satisfactory  interval  ad- 
justment method  requires  re-examination  and  possible 
changes  whenever  another  major  component  of  the 
calibration  control  system  is  changed. 

The  emphasis  here  is  clearly  on  the  economics  of 
choosing  and  establishing  an  interval  adjustment  meth- 
od.   Let  us  not  get  carried  away  by  elegant  mathe- 


matics. Several  plausible  interval  adjustment  methods 
have  appeared  in  various  trade  publications.  Yet  a 
check  with  the  authors  of  several  of  these  a  few  years 
after  publication  revealed  that  the  methods  were  either 
never  implemented  or  were  abandoned  after  a  short 
trial.  They  had  failed  the  crucial  tests  of  practicability 
or  economy  or  both.  In  Appendix  B,  I  describe  an 
interval  adjustment  method  which  has  met  these  tests, 
although  its  theory  is  assailable.  But  it  works  in  the 
environment  for  which  it  was  designed.  It  may  not 
be  suitable  for  adoption  in  other  calibration  control 
systems. 

An  interval  adjustment  method  and  the  resulting 
calibration  intervals  must  above  all  fulfill  the  require- 
ments of  Key  No.  4,  and  be  compatible  with  other 
components  of  the  calibration  control  system.  Once 
these  conditions  have  been  satisfied,  our  attention  may 
then  be  diverted  from  calibration  intervals  toward 
other  problem  areas  within  the  system.  An  impor- 
tant key  towards  Progress  to  Innovative  Metrology  be- 
comes clearly  visible: 

Key  No.  7 — "Feed  the  opportunities"  which  the  cali- 
bration control  system  offers  for  obtaining  optimum 
intervals,  "and  starve  the  problems"  presented  by  sys- 
tem components.2 

4.  Conclusion 

Show  me  a  calibration  control  system  which  achieves 
a  given  quality  level  for  its  measuring  instruments  at 
the  lowest  cost  and  I'll  show  you  calibration  intervals 
which  are  optimized  in  relation  to  all  other  parts  of 
this  system,  and  which  permits  you  to  direct  your 
resources  towards  the  solution  of  other  problems.  If 
your  interval  adjustment  method  reasonably  meets  the 
requirements  of  Key  No.4,  future  effort  will  probably 
yield  higher  returns  when  applied  to  other  problem 
areas. 

Credit  is  due  to  Messrs.  E.  M.  Hicks  and  J.  D. 
Mitchell  of  Autonetics  for  their  many  valuable  com- 
ments and  suggestions. 

Appendix  A.  Sources  of  Uncertainties  of 
Instrument  Quality  Levels 

A.l.  Methods  to  Measure  Instrument 
Quality  Levels 

(a)  At  Recall — This  is  the  method  that  should  not 
be  used  unless  it  has  been  shown  that  there  is  a  known 
relationship  between  the  quality  level  measured  at 
recall  and  that  found  when  the  instruments  are  in  use. 

For  the  quality  level  measured  at  recall  to  bear  a 
meaningful  relationship  to  the  quality  level  of  the  in- 
strument population  in  use,  we  must  first  remove  the 
bias  which  the  calibration  interval  adds  to  the  raw 
ratio  of  "instruments  in  tolerance"  to  the  total  of 
instruments  recalled.  Some  instruments  are  recalled 
for  calibration  much  more  frequently  than  others,  and 
ideally  those  instruments  are  recalled  most  often  which 
need  it  the  most.   The  bias  may  be  removed  by  calcu- 

2  Quote  by  Peter  Drucker,  management  consultant  and  lecturer  at 
NYU  Graduate  School  of  Business. 
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lating  the  percentage  in  tolerance  as  shown  in  table  1, 
using  the  formula 

Percent  in  tolerance  = 

Jiiix  +  n2i2  +  n3i3  +  .  . .  +nkili+ 


X  100 


Qxk  +  Q2i2  +  Qj,  +  ...  +<M+ 

where  is  a  recall  interval  in  weeks  and  Q1(  is  the  total 
number  of  instruments  recalled  at  interval  4> 

Table  1.  The  effect  of  the  bias  of  calibration  intervals 
on  the  apparent  fraction  of  instruments  in  tolerance 
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5 
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70 
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100 

10 

0.80 

80 

5.2 
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416 

100 

20 

0.90 

90 

2.6 

260 
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100 

30 

0.96 

96 

1.7 

173 

166 

1993 

1544 

Actual  (unbiased)  percent  in  tolerance: 
70X5  +  80X10  +  90X20  +  96X30  x  ^  =  ^% 


100  (5  +  10  +  20  +  30) 


Apparent  (biased)  percent  in  tolerance : 
1544 
T99H 


77.5% 


Another  shortcoming  of  measuring  the  instrument 
quality  levels  at  recall  is  that  it  does  not  tell  us  whether 
the  system  is  taking  appropriate  actions  with  regard 
to  individual  instruments  or  instrument  families.  If 
it  allows  instruments  which  are  usually  in  tolerance  to 
be  recalled  very  frequently,  the  percentage  of  instru- 
ments in  tolerance  would  be  deceivingly  high,  conceal- 
ing serious  problems.  The  main  argument,  however, 
against  measuring  the  instrument  quality  level  at  recall 
is  that  the  obtained  figure  per  se  is  totally  irrelevant. 
When  measuring  instruments  are  back  in  the  calibra- 
tion laboratory  for  recalibration,  they  cannot  affect 
production  cost  and  product  quality. 

(b)  In  Use — This  is  the  percentage  that  is  really  of 
interest.  We  want  to  know  what  the  quality  of  meas- 
uring of  those  instruments  is  at  the  time  they  are  being 
used,  when  they  could  have  affected  production  cost 
and  product  quality.  To  know  this,  however,  we  have 
to  go  out  on  the  floor  and  measure  it. 

A  practical  approach  to  measuring  the  quality  level 
of  measuring  instruments  in  use  would  be  to  select  a 
small  sample  at  random  and  calibrate  it.  Using  time- 
honored  methods  of  quality  control,  small  sampling 
methods,  and  control  charts  [4] ,  we  could  in  due  time 
determine  whether  our  calibration  control  system  is 
actually  a  process  "under  control,"  whether  it  is  kept 
at  the  level  which  we  have  determined  as  being  opti- 
mal, and  how  the  calibration  control  system  responds 
to  changes,  "corrective  actions,"  etc. 

This  could  be  done  at  moderate  costs  which  may  be 


more  than  offset  by  savings,  because  we  would  not 
have  to  recall  all  instruments  to  measure  the  quality 
level.  We  could  use  the  recall  strictly  to  operate  on 
the  calibration  control  system,  i.e.,  to  do  only  that 
which  is  necessary  to  maintain  the  quality  level  of 
measuring  instruments  under  control.  These  consid- 
erations may  open  the  field  to  alternatives  to  the  pres- 
ently applied  recall  methods.  However,  regardless  of 
whether  we  continue  applying  the  old  concepts  of 
recall  or  use  new  ones,  we  should  measure  instrument 
quality  levels  "in  use"  rather  than  "at  recall"  to  obtain 
meaningful  figures. 

A.2.  Error  Sources  in  the  Measurement  Process 

To  measure  the  actual  quality  level  of  measuring 
tools  and  instruments,  one  needs,  as  for  all  measure- 
ments, a  standard,  a  comparison  process,  and  decision 
criteria.  These  are  discussed  in  the  following 
paragraphs. 

(a)  Decision  Criteria:  Attributes  versus  Variable 
Data.  The  upper  and  lower  limits  of  an  acceptable 
quality  level  should  be  determined  at  the  time  when 
the  optimum  quality  level  is  determined.  In  theory, 
the  instrument  maintenance  program  should  then  be 
designed  so  that  the  measurement  process  is  in  control 
within  these  limits.  Practically,  however,  upper  and 
lower  control  limits  will  be  a  compromise  between 
what  is  economically  desirable  as  far  as  the  production 
process  is  concerned,  and  what  is  practically  achiev- 
able as  far  as  the  instrument  maintenance  program  is 
concerned. 

Quality  levels  in  instrument  control  systems  are 
usually  given  in  "percent  in  tolerance"  figures.  I  per- 
sonally prefer  the  use  of  "percentage  defectives," 
because  it  is  more  in  line  with  quality  control  lingo 
and  suggests  the  use  of  systematic  quality  control 
methods.  Either  statistic  is  based  on  an  attribute — it 
is  either  in  tolerance  or  out  of  tolerance.  And  that  is 
all  that's  needed  for  the  measurement  and  control  of 
quality  level,  since  the  vast  majority  of  measuring 
instruments  require  the  simultaneous  control  of  a  num- 
ber of  parameters  or  characteristics.  What  counts  in 
the  case  of  determining  common  calibration  intervals 
are  only  the  statistics  which  show  how  often  the  in- 
struments are  out  of  tolerance  (and  perhaps  which 
parameters  are  out  of  tolerance)  not  by  how  much 
they  were  out  of  tolerance. 

Variables  data,  although  more  costly  to  obtain  and 
record,  may  be  needed  to  analyze  the  behavior  of  un- 
reliable instruments  or  instrument  families.  But  these 
should  be  taken  only  when  such  analyses  appear  de- 
sirable. ( It  should  be  remembered  that  we  are  talking 
here  about  the  bulk  control  of  large  quantities  of 
working  instruments,  not  about  instruments  which 
which  must  be  controlled  individually,  like  standards 
[2,3].)  An  enormous  amount  of  variable  data  serves 
no  purpose  other  than  the  satisfaction  of  some  policy 
provisions.  And  that  is  no  good  reason  to  take  such 
data. 

Variables  data  are  somewhat  more  prone  to  error 
than  attributes  data.  Thus,  recording  of  attributes 
data  may  enable  us  to  measure  the  quality  level  of 
measuring  instruments  more  accurately. 
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(b)  The  Standard:  Specifications.  The  standards 
against  which  measuring  instruments  are  tested  are 
usually  their  specifications.  Much  is  being  done  now 
to  make  specifications  more  meaningful;  much  more 
remains  to  be  done.  Problems  commonly  encountered 
in  deciding  what  specifications  to  use  and  what  they 
mean  have  been  discussed  in  detail  elsewhere  [5] .  I 
will,  therefore,  touch  on  some  of  these  problems  only 
peripherally  where  they  have  direct  bearing  on  cali- 
bration intervals. 

The  specification  we  need  to  work  with  in  a  cali- 
bration control  system  must  be  a  realistic  one.  Some 
tolerance  specifications,  as  they  are  given  by  instru- 
ment manufacturers,  are  at  best  optimistic.  Some 
instruments  hold  their  quoted  tolerance,  say  1  percent, 
only  in  the  laboratory:  some  seem  to  hold  their  quoted 
tolerance  only  in  a  random  fashion;  some  others  hold 
it  for  a  short  time  and  require  frequent  recalibration 
to  remain  useful;  however,  some  instruments  hold 
their  manufacturers'  quoted  tolerance  for  very  long 
intervals. 

In  the  case  of  an  instrument  whose  performance 
gradually  deteriorates  with  time,  we  could  select  either 
tight  tolerances  and  short  intervals,  or  loose  tolerances 
with  long  intervals,  or  something  in  between.  Which 
tolerance-interval  pair  should  we  select? 

Generally  speaking,  we  should  select  tolerances  and 
intervals  which  offer  us  the  greatest  return.  The 
optimum  tolerance  specification  depends  on  the  quality, 
cost,  and  behavior  of  the  instrument  as  well  as  the 
cost  of  its  maintenance.  Let  us  take,  for  instance, 
instrument  A  which  costs  $2,000  and  which  is  capable 
of  holding  its  maximum  realistic  tolerance  of  0.1  per- 
cent of  some  characteristic  only  at  the  expense  of  con- 
siderable maintenance  costs  (perhaps  frequent  recali- 
bration), say  $700  a  year.  The  same  instrument  may 
be  capable  of  holding  a  lesser  tolerance,  say  0.15  per- 
cent, easily,  requiring  only  one  annual  calibration  of 
$100  a  year.    Now  let's  look  at  the  alternatives. 

If  we  need  a  0.1  percent  instrument  and  the  next 
better  one,  instrument  B,  costs  $5,000  with  annual 
maintenance  I  including  calibration )  costs  of  $200,  we 
may  be  better  off  with  instrument  A  and  high  mainte- 
nance expenses  and  frequent  recalibrations.  On  the 
other  hand,  there  may  be  a  less  costly  alternative 
available  for  the  0.1  percent  instrument,  but  we  may 
also  need  0.15  percent  instruments  for  which  the 
alternative  would  be  instrument  C,  a  $1,000  instru- 
ment with  $350  maintenance  costs,  including  fairly 
frequent  recalibrations.  In  this  case,  it  may  be  better 
to  use  instrument  A,  assigning  to  it  a  0.15  percent 
tolerance  and  calibration  intervals  of  one  year. 

The  optimum  tolerance  specification,  like  the  opti- 
mum calibration  interval,  can  be  determined  only  in 
the  context  of  economic  alternatives — the  requirements 
of  the  production  process,  the  handling  and  care  of 
the  instruments  in  their  use  environment,  the  training 
of  operators,  etc.  Calibration  intervals  change  when 
specifications  change,  but  specifications  are  functions 
of  variables  outside  the  calibration  control  system. 

Figure  1  illustrates  the  effect  of  doubling  the  specifi- 
cation tolerance  applied  to  a  considerable  number  of 


a  popular  VTVM.  At  15  weeks,  for  example,  the 
percentage  in-tolerance  jumped  from  46  to  86  for  type 
A;  while  for  type  B  it  jumped  from  78  to  93. 

There  is  also  the  problem  of  what  to  include  under 
the  characteristics  which  determine  whether  an  instru- 
ment is  in  tolerance  or  out.  It  may  or  may  not  matter 
whether  an  instrument  with  rated  input  impedance  of 
infinity  has  actually  an  input  impedance  of  50  Mfl, 
whether  an  instrument  with  5  minimum  sensitivity 
has  actually  6  /xV  sensitivity,  whether  a  comparator 
with  rated  contact  pressure  of  10  g  has  only  7  g  con- 
tact pressure.  If  it  does  not  matter,  such  characteris- 
tics should  be  left  out  of  the  consideration  of  an 
instrument's  in-tolerance  condition.  Many,  many  in- 
struments have  been  labeled  out  of  tolerance  for  failure 
of  some  inconsequential  characteristic  to  be  within 
manufacturer's  specifications.  Management  can  do 
much  to  lengthen  calibration  intervals  by  precisely 
defining  which  characteristics  must  be  considered  and 
which  must  be  excluded  when  judging  an  instrument 
out  of  tolerance.  The  question  of  how  to  treat  an 
instrument  with  a  broken  line  cord  or  damaged  line 
plug  may  be  trivial  to  engineers  and  supervision,  but 
not  to  the  technician  who  may  decide  to  call  such  an 
instrument  out  of  tolerance  because  something  was 
wrong  with  it. 

(c)  The  Comparison  Process.  Calibration  proce- 
dures are  widely  considered  as  crutches  for  dimwitted 
calibration  technicians.  Adequately  staffed  calibration 
laboratories  frequently  consider  procedures  unneces- 
sary and  the  respective  Government  requirements  a 
nuisance,  but  this  is  based  on  a  misconception. 

A  calibration  is  a  measurement  process.  A  measure- 
ment deserves  its  name  only  if  it  is  repeatable.  The 
procedure  describes  the  process  and,  therefore,  makes 
it  repeatable.  Two  different  measurement  processes, 
intended  to  measure  the  same  quantity,  mai  by  chance 
yield  the  same  numerical  result.  On  the  other  hand, 
two  measurements  made  by  the  same  process  may  yield 
two  different  results.  No  result  means  much  without 
a  procedure  which  tells  us  how  we  got  that  result. 

The  measurement  method  which  a  procedure  de- 
scribes will  influence  the  magnitude  of  systematic 
errors  in  the  result.  The  number  of  relevant  details 
in  a  procedure  influences  the  magnitude  of  random 
errors.  Therefore,  adequate  calibration  procedures 
are  prerequisites  for  meaningful  interval  adjustment 
methods.  Calibration  intervals  can  be  expected  to 
vary  as  calibration  procedures  change. 

( d )  Policy.  The  policies  governing  a  calibration 
control  system  have  a  marked  effect  on  calibration 
intervals  and  the  methods  for  optimizing  them,  and 
it  is  in  this  area  of  policy  where  management  can 
exert  its  greatest  influence  in  reducing  costs  and  maxi- 
mizing output.  We  have  already  noted  that  the  toler- 
ance specifications  against  which  an  instrument  will 
be  judged  must  be  carefully  defined  for  cost-effective 
interval  adjustment  methods.  But  there  are  other 
policy  elements  which  will  affect  results,  e.g.,  the 
question  of  how  to  treat  completely  inoperable  instru- 
ments, or  instruments  which  are  obviously  malfunc- 
tioning, or  how  to  treat  the  tolerances  of  calibration 
standards. 
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Some  laboratories  have  a  policy  of  subtracting  the 
tolerance  of  the  standard  from  the  tolerance  of  the 
instrument  under  test  if  the  tolerance  ratio  is  larger 
than  1 :4,  for  instance.  One  popular  standard  used  to 
calibrate  the  VTVM's  of  figure  1  provides  only  a  1:2 
ratio  on  the  tolerances  quoted  by  the  manufacturer. 
By  subtracting  the  tolerance  of  the  standard,  one  would 
in  effect  cut  the  tolerance  of  the  VTVM  in  half,  or 
expect  the  instrument  to  perform  within  a  tolerance 
it  was  not  intended  to  meet.  On  the  other  hand,  ignor- 
ing the  tolerance  of  the  standard  would  be  tantamount 
to  increasing  the  instrument's  tolerance  and  increasing 
its  percentage  in-tolerance. 

Other  important  factors  by  which  laboratory  policy 
influences  the  quality  level  for  measuring  instruments 
and  tools  concern  the  generation,  maintenance,  and 
use  of  data  sheets  from  which  quality  decisions  are 
made;  the  motivation  of  laboratory  personnel  to  show 
high  in-tolerance  percentages;  the  motivation  of  lab- 
oratory personnel  to  charge  time  to  uncontrollable 
repair. 

Personnel  selection  and  training  are  still  other 
factors.  In  one  instance,  we  have  found  that  new- 
comers to  the  calibration  laboratory  were  assigned  to 
calibrate  the  common  varieties  of  measuring  instru- 
ments. As  these  technicians  became  more  proficient, 
they  "graduated"  to  more  sophisticated  types  of  meas- 
uring instruments.  The  result  was  that  the  common- 
type  instruments  had  a  lower  quality  level  than  when 
they  were  maintained  by  more  experienced  personnel. 
This  had  a  marked  effect  on  the  overall  quality  level 
of  measuring  instruments,  because  of  the  relatively 
large  number  of  common-type  instruments. 

Appendix  B.  An  Empirical  Interval 
Adjustment  Method 

The  following  describes  an  empirically  developed 
method  for  the  adjustment  of  calibration  intervals  of 
measuring  tools  and  instruments.  It  is  offered  as  an 
example  of  a  method  meeting  all  the  requirements  of 
the  economics,  practicability,  and  speed  of  response 
for  a  given  calibration  control  system.  It  is  not  rec- 
ommended for  adaptation  to  other  calibration  control 
systems  without  careful  analysis  of  compatibility  with 
other  system  components  and  system  objectives.  The 
method  achieves  a  given  quality  level  in  use  within 
reasonably  narrow  bounds  of  uncertainty,  even 
though  the  decisions  concerning  interval  changes  are 
based  on  the  in-tolerance  conditions  of  individual 
instruments  at  recall. 

Exhibit  1  shows  the  Decision  and  Assignment  Tables 
from  which  the  calibration  technician  determines  cali- 
bration intervals.  With  an  instrument  given  him  for 
calibration  is  a  record  of  the  most  recent  tolerance 
codes  for  that  instrument.  These  codes — 1,  3,  or  5 — 
are  defined  below.  He  checks  the  instrument  and 
records  its  incoming  tolerance  code.  Using  the  De- 
cision Table,  he  selects  either  an  asterisk  or  one  of 
the  letters  p,  m,  d.  The  asterisk  means  "Do  not 
change  the  instrument's  current  interval";  a  letter 
refers  him  to  the  Assignment  Table,  from  which  he 
selects  a  new  interval. 


Exhibit  1.  Interval  adjustment  schedule 
DECISION  TABLE 


Previous  Tolerance 

Codes 

Incom 

ing  Tolerance  Code 

CRIS  TREND 

(Left  to  right) 
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5 

111 
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311 
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511 
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#13 

* 

m 
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#15 
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m 
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#31 

* 
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#33 
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#35 

* 

m 
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#51 

* 
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* 

#53 

* 

m 

m 

#55 

* 

m 

m 

ASSIGNMENT  TABLE 

New 

Interval 

Interval 

Current  Interva 

1 

Current 

New 

Current 

New 

Interval  p 

m 

d  Interval 
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m 

d  Interval 

I' 
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5  7 

5 

21 

25 

14 

1') 

37 

12 

25 

33 

6  8 

5 

5 

22 

26 

14 
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38 
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34 

7  9 
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6 

23 

27 

15 
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39 

1 1 

27 

35 

8  10 

5 

7 

24 

28 

15 

22 

40 

16 
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6 

8 

25 

29 

16 

22 

41 
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9 

26 
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17 

23 

42 

48 

28 

38 
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27 

31 
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24 

43 

49 

29 

39 
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7 
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44 

50 

29 

40 
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8 

12 
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33 

20 
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45 

51 

30 

41 
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8 

13 

30 
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20 

27 

46 

52 

31 

41 

H      15  18 

9 

14 
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36 

21 

28 

47t 

52 

32 

42 

~       16  19 

10 

14 

32 

37 

21 

29 

48 
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33 

43 

17  20 

11 

15 

33 

38 

22 

30 

49 

52 
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44 

18  21 

12 
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22 
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50 

52 

35 
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17 

35 

40 

23 

32 

51 

52 

36 

46 

20  24 

13 

18 

36 

11 

24 

32 

52 

52 

37 

47 

*Do  not  change  interval. 
#Any  or  no  entry. 

**Do  not  release  instrument;  contact  supervision  for  instructions, 
tlnstrument  must  be  recalled  at  least  once  a  year  for  service. 
Calibration  schedule,  however,  may  be  determined  as  follows: 
p — Add  10  weeks  if  Interval  2:47. 
m  or  d — Assign  52  weeks  if  Interval  >52. 
The  tolerance  codes  are  defined  as  follows: 

1 — In  tolerance  (All  calibrated  characteristics  of 
the  instrument  were  in  tolerance  as  received,  and  Code 
5  does  not  apply) . 

3 — Out  of  tolerance  (At  least  one  calibrated 
characteristic  was  out  of  tolerance,  and  Code  5  does 
not  apply) . 

5 — Any  of  the  following: 

a.  Out  of  tolerance  condition  indicated  by  the 
instrument's  user  on  the  accompanying  calibration 
requisition  form  and  confirmed  during  calibration. 

b.  Malfunction;  calibration  impossible  prior 
to  repair,  provided  the  required  repair  could  have 
changed  one  or  more  calibrated  characteristics  of  the 
instrument  (excludes  minor  repairs  not  affecting  cali- 
brated characteristics,  like  line  cord  repair,  blown 
fuse,  etc.) . 

c.  Instrument  found  out  of  tolerance  when 
calibrated  to  a  tighter  specification  than  it  was  previ- 
ously calibrated  to. 
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The  interval  adjustment  method  does  not  provide 
for  calibration  intervals  of  less  than  five  weeks,  and 
any  instruments  for  which  a  shorter  interval  is  indi- 
cated are  pulled  out  and  subjected  to  individual 
analysis.  The  method  also  provides  for  a  mandatory 
analysis  of  instrument  types  whose  average  calibration 
interval  is  seven  weeks  or  less,  to  determine  whether 
changes  in  specifications,  calibration  procedures, 
standards,  or  usage  may  improve  the  balance  between 
maintenance  costs  and  instrument  utility.  The  five 
and  seven  week  limits  were  arbitrarily  selected.  We 
feel  that  an  instrument  or  type  which  needs  recalibra- 
tion  so  frequently  is  generally  costing  too  much  in 
periodic  maintenance.  We  have  annual  data-process- 
ing reports  listing  the  average  calibration  interval  of 
each  instrument  type;  a  single  instrument  with  a 
calibration  interval  significantly  shorter  than  the 
typical  average  interval  is  also  subject  to  examination. 
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INTERVALS  BY  EXCEPTION 
Jerry  Glassman 

U.S.  Navy  Metrology  Engineering  Center,  Pomona,  Calif.  91766 

"Intervals  by  exception"  is  a  new  approach  devised  at  the  Navy  Metrology  Engineering  Center 
to  control  the  evaluation  and  adjustment  of  calibration  intervals  for  the  Navy  Calibration  program. 
It  furnishes  management  with  a  powerful  tool  with  which  to  monitor  calibration  intervals  and  assure 
reliability  of  equipment. 

What  are  some  of  the  interesting  features  of  this  new  technique? 

1.  It  enables  management  to  concentrate  on  problem  equipment  by  pinpointing  "dogs"  and  "gems," 
i.e.,  statistically  poor  and  statistically  good  equipments. 

2.  It  allows  management  to  select  "optimum"  intervals  based  on  automatically  derived  reliability 
tradeoffs. 

3.  It  motivates  laboratory  personnel  by  providing  an  information  feedback  loop  for  problem 
instruments. 

4.  It  lowers  operating  costs  by  funneling  analysis  and  engineering  effort  into  the  areas  of  greatest 
needs  and  most  promising  returns. 

5.  It  lowers  costs  by  lengthening  intervals  on  family  types  and  stabilizing  reliability  levels. 

6.  It  provides  automatic  computer  monitoring  of  all  decisions  (cause-effect  feedback) . 

This  system  is  presented  with  charts  that  illustrate  its  operation,  outputs,  and  manage- 
ment-oriented controls.  The  presentation  includes  data  analyses,  adjusting  of  intervals,  detection  of 
"dogs"  and  "gems,"  resultant  changes  in  system  operating  characteristics,  underlying  mathematical 
assumptions,  and  mathematical  models. 

Key  words:  Calibration  intervals;  exponential  distribution;  reliability,  systems  effectiveness,  tradeoffs. 


1.  Introduction 

The  manager  of  a  calibration  laboratory  is  faced 
with  a  multitude  of  decisions,  many  of  whose  effects 
can  influence  overall  operations  for  years  to  come. 
The  decision  to  change  a  calibration  interval  can  affect 
numerous  system  operating  characteristics,  such  as: 

a.  equipment  reliability/maintainability, 

b.  laboratory  operating  costs, 

c.  repair  time, 

d.  operational  schedules, 

e.  calibration  time, 

f.  manpower  distribution, 

g.  equipment  availability, 

h.  personnel  motivation. 

The  complexity  of  the  interactions  of  these  impor- 
tant system  characteristics  makes  it  incumbent  on  the 
manager  to  avail  himself  of  simple  decision-making 
tools.  Sophisticated  system-effectiveness  techniques 
are  necessary  to  generate  the  tradeoff  alternatives 
required  by  management.  However,  the  decision- 
making apparatus  and  management-oriented  monitor- 
ing functions  should  provide  complete  visibility  and 
control,  and  immediate  feedback  of  system  effects. 

An  analytical  method  has  been  devised  to  evaluate 
equipment  calibration  intervals  and  provide  justifica- 
tion for  recommending  changes  based  on  the  cumu- 


lated results  of  past  calibrations.  The  approach  is 
rigorously  developed  and  contains  several  unique 
features  not  found  in  other  systems.  Additionally, 
this  system  provides  management  with  tools  with 
which  to  make  difficult  optimization  decisions  without 
referring  to  the  underlying  mathematical  processes. 
The  tradeoffs  are  made  available  to  him,  and  the 
predicted  system  effects  are  provided  for  several 
alternatives. 

A  decision-making  system  is  effective  only  if  it  is 
used  to  make  decisions.  It  is  a  valuable  management 
tool  if  it  is  easy  to  use  and  provides  overall  system 
visibility  and  direct  feedback  of  effects.  "Intervals 
by  exception"  offers  new  management  decision-making 
approach  to  calibration  interval  determination  and 
interpretation  without  compromising  mathematical 
rigor. 

2.  Systems  Based  on  Model  Number 
or  Serial  Number 

Most  calibration  interval  analysis  systems  fall  into 
one  of  two  categories: 

1.  Intervals  by  model  number.  All  equipments  of 
the  same  model  number  are  treated  as  if  homogeneous, 
and  a  calibration  interval  is  established  for  the  model 
number  family. 
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2.  Intervals  by  serial  number.  Each  unit  of  serially 
numbered  equipment  is  assumed  to  function  according 
to  its  own  unique  operating  characteristics,  and  a 
calibration  interval  is  established  for  each  individual 
equipment. 

Both  of  these  general  approaches  betray  serious 
shortcomings  in  application.  If  the  deficiencies  are 
not  eliminated,  the  resulting  decisions  are  seldom  opti- 
mum and  are  often  erroneous.  If  the  deficiencies  are 
eliminated,  the  systems  become  overcomplicated,  and 
lose  the  characteristics  that  initially  made  them 
attractive. 

Examples  of  deficiencies  related  to  calibration  in- 
terval systems  based  on  model  number  or  serial 
number  are  as  follows: 

Case  1.    Intervals  by  Model  Number 

a.  Different  manufacturers  might  have  produced 
equipments  having  the  same  model  number. 

b.  Different  fabrication  methods  might  be  used  for 
equipments  of  the  same  model  number. 

c.  Equipments  might  have  experienced  design  modi- 
fications without  model  number  changes. 

d.  Cumulative  operating  time  is  different  for  equip- 
ments of  the  same  model  number  installed  at  various 
dates. 

e.  Operational/environmental  differences  may  exist 
for  equipments  of  the  same  model  number. 

These  and  other  factors  can  cause  significantly 
varying  operating  effects  that  should  be  considered 
or  removed.  The  usual  treatment  entails  subgrouping 
the  equipments  by  significant  factor  levels,  and  then 
establishing  separate  intervals  for  each  subgroup. 
Figure  1  is  an  example  of  the  multiplicity  of  sub- 


groups that  arise  when  as  few  as  three  significant 
factors  are  considered.  As  can  be  seen,  the  magnitude 
of  the  problem  rapidly  becomes  unmanageable  as  the 
number  of  significant  factors  increases.  The  serial 
number  system  evolves  when  it  becomes  advisable  to 
consider  all  the  factors  contributing  to  the  equipment 
operation. 

Case  2.    Intervals  by  Serial  Number 

a.  Initial  establishment  of  intervals  for  new  equip- 
ments is  at  best  an  educated  guess;  when  the  serial 
number  philosophy  is  used,  the  historical  data  avail- 
able is  of  little  use  for  establishing  intervals. 

b.  The  amount  of  calibration  data  accumulated  for 
one  serial  number  is  generally  insufficient  for  mean- 
ingful statistical  analysis.  Therefore,  the  assurance 
of  equipment  reliability  is  essentially  impossible  for 
most  equipments. 

c.  Management  control  systems  based  on  serial 
number  data  are  completely  meaningless  in  a  system 
effectiveness  context;  i.e.,  the  resultant  decisions  can- 
not be  related  to  other  system  parameters,  nor  are 
meaningful  tradeoffs  available. 

d.  Administrative  and  routine  controls  are  difficult. 
As  a  result  of  the  above  shortcomings,  serial-based 

interval  evaluation  systems  tend  to  substitute  ritualistic 
and  arbitrary  rules  for  rigorous  statistical  methods. 

3.  The  New  Approach 

Recognition  of  these  discrepancies  precipitated  an 
investigation  of  new  approaches  to  calibration  data 
analysis  and  interpretation  in  this  vital  area  of  metrol- 
ogy.   "Intervals  by  exception"  is  the  result  of  this 
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Ficure  1.  System  complexity  induces  conversion  from  a  model  number  to  a  serial  number  system  when  the 

number  of  factors  considered  is  increased. 
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investigation  conducted  at  the  Navy  Metrology  Engi- 
neering Center  for  application  to  the  Navy  calibration 
program. 

Imagine  a  group  of  instruments  of  a  particular 
model  number  designation.  Ideally,  these  equipments 
should  be  capable  of  achieving  a  specified  reliability, 
when  used  as  required  for  a  given  time  between  cali- 
brations. However,  because  of  the  diverse  influences 
of  factors  affecting  equipment  reliability,  each  serial- 
ized equipment  of  this  model  may  follow  a  different 
pattern  of  failure. 

The  random  interaction  of  factors  causes  most 
equipment  to  be  influenced  by  a  combination  of  ele- 
mentary negative  effects.  Thus,  compensatory  factor- 
levels  tend  to  balance  and  the  residual  effect  varies 
slightly  around  the  average.  Hence,  the  majority  of 
equipments  have  similar  operating  characteristics. 
The  only  equipments  that  display  operating  charac- 
teristics significantly  different  from  the  average  are 
those  having  a  preponderance  of  negative  factor-levels 
or  positive  factor-levels.  These  "exceptions"  in  the 
model  number  group,  and  only  these,  need  be  con- 
sidered for  special  interval  treatment. 

To  illustrate  the  effect  of  this  random  interaction, 
suppose  that  the  ratings  and  weights  shown  in  table  1 
are  arbitrarily  assigned  to  items,  A,  B,  C  at  factor- 
level  1  in  figure  1,  to  items  1,  2,  3  at  factor-level  2, 
and  to  items  Z,  Y,  X,  V,  U  at  factor-level  3.  For  ex- 
ample, subgroup  A1Z  consists  of  equipments  manu- 
factured by  company  A,  using  production  method  1, 
and  having  an  operating  age  of  Z  months;  its  rating 
would  be  —3  as  shown  in  figure  2.  At  the  other 
extreme,  subgroup  C2V  rates  as  +3.  Each  of  these 
two  equipment  subgroups  would  probably  have  an 
operating  reliability  deviating  significantly  from  the 
average,  one  much  poorer  and  one  much  better.  We 
designate  such  equipments  as  "dogs"  and  "gems," 
respectively. 

Table  1.  Assumed  factors,  levels,  level  ratings,  and 
 weights  for  a  hypothetical  model  number. 


Factors 

Levels 

Ratings 

Weights 

Manufacturer 

A 

-1 

B 

• 

0 

C 

+ 

+  1 

Production  method 

1 

-1 

2 

+ 

+  1 

3 

• 

0 

Operating  age 

Z 

-1 

Y 

• 

0 

X 

• 

0 

V 

+ 

+1 

u 

• 

0 

Ratings  legend: 

+  better  than  average 
•  average 

—  poorer  than  average 


As  can  be  seen  in  figure  2,  most  of  the  subgroups 
(10  of  14)  have  ratings  of  —1,  0,  or  +1;  these  would 
probably  have  near-average  operating  reliability,  and 


may  be  considered  to  be  "equivalent  equipments." 
Subgroups  with  ratings  —2  or  +2  are  borderline 
cases  which  may  or  may  not  show  exceptional 
reliability. 


C3U 

A3X 

B3X 

B3V 

A2Z 

A3V 

B2X 

A1Z 

A1  Y 

A1V 

A2Y 

B2Y 

C2U 

C2V 

-3          -2          -1            0         +1          +2  +3 
SUBGROUP  RATING 

Figure  2.  Hypothetical  histogram  of  equipment 
subgroups  by  subgroup  rating. 

Figure  3  illustrates  the  simplicity  in  concept  of 
"intervals  by  exception."  Compare  this  to  the  com- 
plexity and  multiplicity  of  subgroups  generated  in 
figure  1. 


MODEL  NUMBER  GROUP 


DOGS 


EQUIVALENT  EQUIPMENT 


GEMS 


Figure  3.  Simplified  representation  of  classification 
based  on  intervals  by  exception. 

"Intervals  by  exception"  is  a  new  approach  devised 
to  evaluate  and  adjust  calibration  intervals  based  on 
a  compromise  between  the  simplistic  model  number 
and  the  ritualistic  serial  number  systems.  Not  only 
does  it  allow  for  statistical  analysis  of  the  majority 
of  instruments  by  model  number,  but  it  pinpoints 
those  equipments  requiring  special  attention. 

4.  Norms,  Dogs,  and  Gems 

In  general,  equipment  belonging  to  model  number 
families  exhibit  similar  operating  characteristics  if 
manufactured  and  operated  under  similar  conditions. 
However,  within  any  model  number  family,  minor 
differences  exist  that  cause  particular  serialized  items 
to  perform  significantly  better  or  worse  than  the 
model  number  norm.  Differences  in  manufacturers, 
production  method,  operating  time,  or  other  factors 
can  cause  these  variations.  By  isolating  (excepting) 
these  significantly  deviating  items,  realistic  calibra- 
tion intervals  can  be  determined  for  the  remaining 
conforming  equipment  comprising  the  majority  of  the 
model  number  group. 
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Equipments  that  perform  significantly  differently 
than  the  norm  are  called  "dogs"  or  "gems"  according 
to  whether  they  are,  respectively,  poorer  than  or  better 
than  the  group  average.  These  are  the  exceptions  in 
the  model  number  group: 

1.  Dogs — individual  serialized  items  that  yield  a 
statistically  poorer  out-of-tolerance  rate  than  the 
family  average. 

2.  Gems — individual  serialized  items  that  yield  a 
statistically  better  out-of-tolerance  rate  than  the  fam- 
ily average. 

Analyses  of  these  exceptional  equipments  provide  a 
valuable  potential  for  generating  cost  savings,  im- 
proving equipment  operation,  and  disclosing  new 
areas  of  engineering  knowledge  related  to  equipment 
performance  characteristics. 

5.  Mathematical  Assumptions  and  Tests 

The  basic  direction  of  "intervals  by  exception"  is 
toward  management-oriented  control  and  monitoring 
features.  However,  there  is  a  mathematical  basis  for 
the  tradeoffs  and  decision  criteria.  The  detailed 
mathematical  models  and  statistical  tests  that  are 
used  are  contained  in  the  Appendix. 

The  following  mathematical  assumptions  are  made 
in  applying  this  technique  at  Navy  Metrology  Engi- 
neering Center: 

1.  The  times-to-out-of-tolerance  are  exponentially 
distributed  according  to  the  function  ke~kt. 

2.  The  probability  of  out-of-tolerance  increases 
monotonically  as  the  time  between  calibrations  is  in- 
creased for  a  given  equipment. 

3.  Failure  characteristics  for  all  equipments  of  a 
given  model  number  are  described  by  the  same  time- 
to-out-of-tolerance  function,  except  for  a  small  per- 
centage of  "dogs"  and  "gems." 


The  analyses  of  the  records,  for  a  given  family  of 
equipments  and  a  given  calibration  interval,  follow 
the  general  guidelines  described  below: 

1.  Data  are  purged  of  irrelevant  and  erroneous 
entries;  i.e.,  obvious  mistakes  and  human  errors  are 
removed  from  the  data  base. 

2.  The  data  are  adjusted  using  criteria  and  re- 
quirements based  on  known  or  assumed  system  effects. 
For  example,  the  tails  of  the  time-to-failure  distribu- 
tion are  truncated  to  avoid  using  nonrepresentative 
or  heterogeneous  data;  the  times  to  out-of-tolerance 
are  estimated,  because  only  times-to-discovery  of  out- 
of-tolerance  are  available,  rather  than  actual  times-to- 
driftout. 

3.  Statistical  tests  of  significance  are  applied  to 
detect  changes  in  equipment  operation.  This  requires 
computer  grouping  of  data  by  calendar  times  and  com- 
puter testing  of  the  consistency  of  the  out-of-tolerance 
rates.  If  a  statistically  significant  change  is  detected, 
only  the  most  recent  consistent  data  are  utilized  for 
analyses.  Therefore,  the  data  used  for  analyses  are 
homogeneous.  The  effects  of  design  modification, 
procedural  changes,  and  environmental/operational 
differences  are  discovered,  isolated,  and  analyzed 
separately. 

4.  The  cumulative  Poisson  probability  distribution 
is  used  to  test  the  relationship  of  the  parameter  for  an 
individual  serial  number  to  the  average  for  the  model 
number.  This  detects  statistical  outliers;  i.e.,  serial- 
numbered  "dogs"  and  "gems." 

5.  An  estimate  of  the  out-of-tolerance  rate  is  ob- 
tained for  the  remaining  group  of  equivalent 
equipments. 

6.  Predictions  of  the  probabilities  that  the  equip- 
ments will  remain  in-tolerance  are  made  for  several 
different  calibration  intervals.    This  provides  trade- 


FlGURE  4.  Sample  of  a  calibration  interval  analysis,  01/07/70. 
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Figure  5.  Flowchart  of  closed-loop  feedback  system  for  intervals  by  exception. 
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ofTs  enabling  management  to  make  the  decision  for 
adjusting  intervals  with  total  visibility  of  the  effect 
on  reliability  for  the  model  number  population. 

7.  New  methods  are  being  developed  to  derive 
reliability/effectiveness  and  interval/cost  trade  offs  for 
determining  optimum  calibration  practices.  These 
trade-offs  will  allow  management  to  consider  the  sys- 
tem effects  of  out-of-tolerance  equipment  and  evolve 
criteria  for  repairing  versus  replacing  items. 

8.  An  original  approach  is  being  developed  for  esti- 
mating and  applying  confidence  limits  to  reliability/ 
interval  decisions.  Some  equipments  perform  so  well 
that  few  failures  develop  in  them.  Current  methods 
do  not  provide  a  high  degree  of  assurance  for  extend- 
ing intervals  on  these  kinds  of  "gems."  The  new 
confidence-limit  approach  allows  extension  of  intervals 
with  increased  statistical  assurance. 

6.  Results 

It  was  necessary  to  design  new  computer  output 
formats  for  the  calibration  interval  analysis  so  that 
the  results  could  be  efficiently  presented  to  manage- 
ment. These  also  provide  prompt  feedback  of  de- 
cisions to  cognizant  user  activities,  and  facilitate 
engineering  analyses  of  "dogs"  and  "gems." 

Figure  4  is  a  sample  presenting  the  purged  calibra- 
tion data,  the  estimated  reliability/interval  trade-offs, 
and  the  "dog"  and  "gem"  serial  numbers.  Note  that 
the  data  for  "equivalent  equipment"  includes  only  the 
number  reported  and  the  number  out-of-tolerance 
(00T)  ;  reports  of  damaged  and  inoperative  equip- 
ments do  not  appear. 

The  major  computer  analyses  subprograms  are: 
a.  Data  truncation. 


b.  Data  adjustment. 

c.  Statistical  tests  for  significant  differences  be- 
tween data  groups  and  for  the  establishment  of  the 
data  base. 

d.  Determination  of  "dogs"  and  "gems." 

e.  Calculation  of  reliability/calibration  interval 
trade-off  alternates. 

Figure  5  is  a  flowchart  of  the  closed-loop  feedback 
system  implemented  for  "intervals  by  exception." 

Table  2  is  an  example  of  the  input  to  management 
showing  the  interval/reliability  trade-offs  for  selected 
values  of  out-of-tolerance  rate. 


OUT-OF-TOLERANCE 
RATE,  K 

CALIBRATION 
INTERVAL 

RELIABILITY 

.016 

7 

.894 

10 

.852 

14 

.799 

18 

.750 

.017 

6 

.903 

10 

.844 

13 

.802 

17 

.749 

.018 

6 

.898 

9 

.850 

12 

.806 

16 

.750 

Table  2.  Partial  table  of  tradeoffs  between  estimated 
calibration  intervals  and  reliability  values,  by  out-of- 
tolerance  rate,  k. 
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MONTHS-BE  TWE  EN -CALIBRATIONS 

FIGURE  6.  Distributions  of  current  and  projected  intervals  for  60  families  of  equipments. 
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7.  Conclusions 

'  A  new  philosophy  has  been  presented  for  inter- 
preting calibration  intervals.  This  philosophy,  called 
"intervals  by  exception,"  has  been  developed  into  an 

(i  analytical  system  for  evaluating  calibration  data. 
Statistical  techniques,  resourcefully  applied,  constitute 
the  framework  of  this  system.  The  system  enables 
management  to  confidently  make  interval  adjustment 

t  decisions  with  complete  visibility  of  their  probable 

:  effects  on  equipment  reliability. 

This  system  is  being  used  at  the  Navy  Metrology 
Engineering  Center  to  evaluate  calibration  data  and 
evolve  decision  criteria  on  which  to  change  intervals. 

i  A  test  program  was  conducted  on  60  equipment  types 
submitted  for  calibration  at  short  intervals.  Optimum 
decisions  were  made  for  these  60  equipments;  some 
intervals  were  increased,  some  decreased,  and  some 
not  changed.  The  average  interval  for  these  60  was 
increased  from  6.8  months  to  8.2  months.  Figure  6 
shows  the  exponential  probability  distribution  of  the 
60  projected  intervals  superimposed  on  the  distribu- 
tion of  the  60  intervals  currently  in  use  (equal  num- 
bers at  3,  4,  6,  9  and  12  months).  As  can  be  seen, 
there  is  a  significant  shift  (increase)  in  the  mean. 
Also,  there  are  many  more  instruments  at  longer 
intervals  on  the  projected  curve,  and  the  percentage 
out-of-tolerance  has  been  reduced  from  30.1  to  15. 
This  is  clear  evidence  of  the  long  range  (recurring) 
cost  improvements  generated  as  a  result  of  using 
"intervals  by  exception." 

It  has  been  demonstrated  that  meaningful  results 
can  be  obtained  by  approaching  a  problem  in  an  orig- 
inal manner.  It  is  hoped  that  this  effort  acts  as  a 
catalyst  for  imaginative  inquiry  into  other  problems 
in  metrology  in  addition  to  its  being  used  in  its  own 
right. 
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8.  Appendix 
8.1.  Reliability 

It  is  hypothesized  that  the  probability  that  an  in- 
strument remains  in  tolerance  is  a  function  of  the 
elapsed  time  since  it  was  last  calibrated.  This  prob- 
ability, called  reliability  R(t),  is  equal  to  the  comple- 
ment of  the  cumulative  probability  distribution  P{t) 
for  times-to-out-of-tolerance  for  all  instruments  of  that 
family;  i.e.,  R(t)  =  l-P(t). 

The  underlying  distribution  of  times-to-out-of-tol- 
erance is  generally  assumed  to  be  an  exponential  of 
the  form. 

f(t)  =ke~kl,         0^  t<  oo, 
=  0       ,  otherwise 

where  k  is  the  out-of-tolerance  rate;  k>0,  and  e  is  the 
base  of  natural  logarithms.  The  cumulative  distribu- 
tion function,  which  represents  the  probability  that 
an  equipment  goes  out-of-tolerance  before  time  t  is 


P(t)  =  fj(t)dt=  (ke-ktdt=  l-e-kl 

The  probability  that  an  equipment  remains  in-toler- 
ance  for  a  time  interval  equal  to  t  is 

R(t)  =  J*f(t)dt=  l-JJ{t)dt  =  e-ki 

where  R{t)  is  the  reliability.  It  can  be  seen  that 
R(t)  +  P(t)  =  1.  For  a  more  detailed  discussion, 
see  reference  [l].1 

8.2.  Data  Consistency 

If  equipment  of  a  given  model  number  undergoes 
design,  procedural,  or  operational  revision,  its  out- 
of-tolerance  rate  may  change.  Only  relevant  calibra- 
tion data  should  be  utilized  when  decisions  are  made 
affecting  the  current  configurations  and  uses  of  equip- 
ments. A  statistical  test  is  performed  to  detect  signifi- 
cant changes  in  out-of-tolerance  rates  for  data  grouped 
by  calendar  time. 

The  probable  occurrence  of  out-of-tolerance  reports 
in  any  specified  period  can  be  mathematically  described 
by  a  Poisson  process  with  parameter  equal  to  the  ex- 
pected number  of  out-of-tolerance  reports. 

'(   '  ~  Xl 

where  f{X)  is  the  probability  of  obtaining  X  out-of- 
tolerance  reports,  k  is  the  family  out-of-tolerance  rate, 
kt  is  the  expected  number  of  out-of-tolerance  reports, 
e  is  the  base  of  natural  logarithms. 
An  exact  F-test  of  equivalency  for  two  sets  of  Poisson- 
generated  data  on  out-of-tolerance  reports  per  unit 
time  is  derived  in  reference  [2]  and  shown  below: 

F1_,(2(X,+  1),2J1) 

where  Xx  and  X2  are  the  number  of  out-of-tolerance 
reports  for  any  two  groups  of  data,  Fj  and  T2  are  the 
corresponding  accumulated  operating  times  for  these 
groups,  and 

7\  TV 

8.3.  Detecting  Dogs  and  Gems 

Tests  of  significance  are  performed  to  detect  sta- 
tistical "dogs"  and  "gems"  by  comparing  data  for  each 
individual  instrument  with  the  model  number  failure 
rate  estimate.  The  following  computations  are  per- 
formed for  each  instrument: 

1.  The  expected  number  of  out-of-tolerance  reports 
for  a  given  serial  number  with  operating  time  Tg  is 
calculated: 

kTg  =  TgZri/ZTi 

where  k  is  the  estimated  out-of-tolerance  rate  for  the 
model  number  family;  r,  is  the  cumulative  number  of 


1  Figures  in  brackets  indicate  the  literature  references  at  the  end  of 
this  paper. 


71 


out-of-tolerance  reports  for  the  ith  serial  number,  Ti 
is  the  cumulative  operating  time  for  the  ith  serial  num- 
ber. 

2.  The  kTg  (expected  number  of  out-of-tolerance  re- 
ports for  the  given  serial  number)  is  compared  to  the 
Tg  (observed  number  of  out-of-tolerance  reports  for 
the  given  serial  number). 

3.  If    kTg    <  Tg, 

_  _^e-^g[kTgV 


is  calculated. 

4.  If  P.i  =  0.05,  the  serial  number  being  tested  is  a 
statistical  "dog." 


5.  If  kTg  >  rg, 


is  calculated. 

6.  If  PB  =  0.05,  the  serial  number  is  a  statistical 
"gem." 
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OPTIMIZING  CALIBRATION  INTERVALS 
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This  paper  describes  a  two-method  system  for  using  simple  attributes  calibration  data  to  adjust  and 
optimize  calibration  intervals.  Each  method  is  used  only  where  its  superior  characteristics  predomi- 
nate; the  combination  of  methods  produces  a  total  system  which  exploits  the  advantages  of  each 
without  having  to  suffer  with  the  disadvantages.  The  "fixed  interval  through  data"  method  is 
employed  where  there  are  at  least  27  bits  of  data,  the  applicaion  being  unique  in  that  the  data  is 
treated  as  a  random  sample  from  an  infinite  population,  sampling  variations  are  accounted  for,  and 
all  decisions  to  extend  calibration  intervals  are  characterized  by  statistical  validity.  When  there  are 
only  from  5  to  26  bits  of  data,  statistical  validity  is  not  possible  and  the)  "fixed  interval  through 
engineering  intuition"  method  is  utilized.  However,  intuition  and  judgment  are  applied  only  to  a  set 
of  decision  rules,  and  strong,  specific  data  support  is  required.  The  system  has  not  been  extended  to 
less  than  5  bits  of  data,  although  there  would  seem  to  be  no  problem  in  so  doing.  Net  results  in  the 
first  18  months  of  use  are  495  changes  involving  34,200  units,  with  annual  savings  of  24,100  technician 
hours,  and  with  no  noticable  deterioration  in  overall  instrument  reliability. 

Key  words:  Data  support;  fixed  calibration  interval;  instrument  reliability;  statistical  validity; 
two-method  optimization. 


1.  Introduction 

The  combination  system  of  adjusting  and  optimiz- 
ing calibration  intervals  described  herein  was  de- 
veloped over  a  period  of  about  two  years  in  the  Lock- 
heed Measurements  and  Standards  Laboratory  (MSL). 
The  "fixed  interval  through  data"  method  was  worked 
out  first,  and  after  six  months  implementation  it  was 
evident  that  the  method  is  highly  satisfactory  as  ap- 
plied to  instrument  groups  for  which  a  substantial 
amount  of  data  is  at  hand.  However,  it  also  became 
obvious  that  the  method  was  either  woefully  inade- 
quate or  totally  unusable  for  families  for  which  few 
data  are  available,  and  that  a  satisfactory  means  of 
handling  the  latter  was  badly  needed.  Out  of  this 
need  evolved  the  second  method  and  the  combination 
of  the  two  has  quite  satisfactorily  solved  a  problem  of 
very  long  standing. 

2.  Definitions 

Operative / Inoperative.  An  instrument  is  operative 
when  all  performance  parameters  are  functioning  and 
capable  of  yielding  data  or  information.  An  instru- 
ment is  inoperative  when  one  or  more  performance 
parameters  are  disabled  and  incapable  of  yielding  data 
or  information. 

Operational  Reliability.  (R0)  for  a  particular  group 
of  instruments,  over  a  given  period  of  time,  is  the 
ratio  of  the  number  of  operative  units  to  the  sum  of 
operative  and  inoperative  numbers  of  units. 

In-tolerance / Out-oj -tolerance.  An  instrument  is  in- 


tolerance when  all  performance  parameters  are  func- 
tioning within  specifications.  An  instrument  is  out- 
of-tolerance  when  one  or  more  performance  parame- 
ters are  not  functioning  within  specifications. 

ln-tolerance  Reliability.  (R,)  for  a  sample  of  units 
submitted  for  calibration  is  the  ratio  of  the  number 
in-tolerance  to  the  total  number  operative. 

Actual  In-tolerance  Reliability.  (Ra)  is  unknown 
except  where  data  are  at  hand  for  an  entire  population, 
and  no  sampling  is  involved.  Ri  is  an  estimate  of  Ra, 
and  statistical  limits  for  Ra  with  any  desired  degree  of 
confidence  can  be  determined  from  the  data  used  to 
calculate  Ri.  Although  Ra  is  unknown,  its  limits  and 
the  degree  of  confidence  associated  with  the  limits  are 
highly  useful. 

3.  Alpha-Code  Data  Base 

The  basic  source  of  reliability  information  is  the 
alpha-code  marking,  directly  related  to  operational  and 
in-tolerance  reliability.  The  alpha-code  is  a  single 
letter  recorded  by  a  technician  on  the  Service  Report 
and  it  provides  significant  information  on  the  as-re- 
ceived condition,  reason  for  submission,  and  work  per- 
formed. The  alpha-codes,  instrument  categories  to 
which  they  apply,  and  category  definitions  are: 

A — ACCEPTANCE  —  applicable  to  instruments 
which  are 

•  New  or  serviced  for  the  first  time 

•  Returned  from  vendor,  storage,  or  held  for 
parts 

•  Put  into  hold  for  manual  status 
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•  Submitted  by  any  facility  external  to  Bay 
Area  MSL 

•  Placed  in  calibration  storage  without  calibra- 
tion 

•  Sent  by  In-place  to  an  MSL  laboratory 

D — DAMAGED — applicable  to  instruments  which 
show  evidence  of  physical  damage  or  opera- 
tional misuse. 

F— INOPERATIVE  -  -  applicable  to  instruments 
showing  one  or  more  performance  parameters 
incapable  of  yielding  information. 

C— OPERATIVE  IN-TOLERANCE  -  -  applicable 
to  instruments  with  all  performance  parameters 
functioning  within  specifications. 

0— OPERATIVE  OUT-OF-TOLERANCE  —  appli- 
cable to  instruments  with  one  or  more  perform- 
ance parameters  not  functioning  within  specifi- 
cations. 

4.  Reliability  Computations 

Figure  1  shows  the  logic  and  procedure  used  by  the 
technician  in  assigning  an  alpha-code  to  the  instru- 
ments submitted  to  MSL. 

P  —  average  population  of  an  instrument  group  over 
a  number  of  months,  m. 

N  =  Pm  =  total  number  of  units  submitted  to  MSL 
in  m  months. 


reliability  Rn  and  in-tolerance  reliability  Ri  are  com- 
puted as: 


A+D 


Ri  = 


m 


A+D 


C+0 


Y-0  C 
Y  Y 


5.  Fixed  Interval  Through  Data 

After  literature  search,  discussion  with  our  labora- 
tory personnel  and  customer  representatives,  and  ex- 
amination of  considerable  data,  the  following  criteria 
for  extension  of  calibration  intervals  were  set: 

•  Ri  must  be  95  percent  minimum  and  R0  must  be 
92  percent  minimum. 

•  At  the  new  interval,  the  probability  that  Ra  is  at 
least  92  percent  must  be  80  percent  or  more. 

Originally,  90/90  percent  was  chosen  for  Ra  rather 
than  92/80  percent  because  90  percent  is  the  midpoint 
of  the  85  to  95  percent  in-tolerance  acceptability  range. 
However,  one  of  our  customers  uses  an  80  percent 
confidence  factor  almost  exclusively  in  reliability  cal- 
culations, so  in  order  to  conform  to  their  normal  prac- 
tice the  confidence  factor  was  dropped  to  80  percent 


Acceptance 


Inoperative  Out-of-tolerance 
F 


D 

Damaged  In-tolerance 

Figure  1.  Logic  for  assignment  of  alpha-code. 


These  codes  are  mutually  exclusive,  so  that  there  is 
one  and  only  one  correct  alpha-code  for  each  item. 
The  only  situation  where  the  technician  may  have  a 
difficult  judgment  is  in  determining  whether  an  instru- 
ment has  suffered  physical  damage  or  has  been  mis- 
used in  operation.  It  should  be  noted  that  the  in- 
tolerance/out-of-tolerance  determination  is  made  by 
comparing  a  quantitative  measurement  with  specified 
parameter  limit (s)  and  that  each  code  provides  sub- 
stantial information  on  what  the  instrument  is  and 
what  it  is  not.  For  example,  the  code  "C"  provides 
information  that  the  instrument  was  operative,  in-tol- 
erance, had  not  been  subjected  to  physical  abuse  or 
operational  misuse,  and  was  not  in  any  one  of  the 
conditions  classified  as  ACCEPTANCE. 

The  estimates  of  reliability  are  based  on  the  num- 
bers of  instruments  accumulated  over  a  stated  period 
in  these  various  alpha-code  categories.  Operational 


and  the  reliability  figure  raised  to  92  percent  to  com- 
pensate. Figure  2  is  the  chart  of  in-tolerance  reliabil- 
ity versus  sample  size  available,  used  for  determina- 
tion of  permissible  calibration  interval  extensions  in 
accordance  with  the  criteria  above.  Zone  0  permits 
no  change,  Zone  1  permits  an  increase  of  one-third, 
Zone  2  permits  an  increase  of  50  percent,  and  Zone  3 
permits  the  interval  to  be  doubled.  (Extensions  from 
6  to  12  or  12  to  24  are  not  permitted  because  of  other 
restraints. ) 

A  minimum  of  27  bits  of  data  is  required  to  satisfy 
the  criteria  and  provide  statistical  validity.  The  27- 
bit  minimum  is  a  function  of  the  Ra  92/80  percent 
criteria  and  will  be  larger  or  smaller  as  these  percent- 
ages are  increased  or  decreased.  An  important  point 
with  respect  to  the  criteria  is  that  inoperatives  (mal- 
functions or  failures)  are  not  ignored  as  they  are  in 
many  calibration  interval  schemes.     R„  must  be  a 
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Figure  2.  Chart  for  determination  of  permissible 
extensions  in  calibration  intervals. 

minimum  of  92  percent  and  thus  acts  as  a  permissive/ 
restraining  factor  in  extension  of  calibration  intervals 
but  does  not  influence  the  magnitude  of  the  extension 
since  it  has  no  influence  on  Ra. 


0.00  IO0  1.50 
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Figure  3.  Two-step  process  for  determination  of  zone 
boundary  lines  in  figure  2. 

Calculation  of  zone  boundary  lines.  The  method  used 
to  calculate  the  zone  boundary  lines  of  figure  2  is  a 
straightforward  two-step  process  involving  elementary 
reliability  statistics  theory.  Figure  3  illustrates  the 
process.  First,  an  R,  at  1.00  t  (the  current  calibra- 
tion interval)  is  assumed  and  noted  as  point  A.  Point 
B  is  calculated  on  the  assumption  that  the  failure1 
rate  k  is  small  and  constant  within  bt  (where  b  is  1.33, 


1  In  this  discussion  the  term  "failure"  refers  to  an  out-of-tolerance 
condition,  i.e.,  failure  to  meet  accuracy  specifications,  rather  than  an 
operating  failure  where  there  is  a  complete  malfunction.  When  Ri  is 
high  and  failure  rate  is  low  (as  is  generally  the  case),  it  is  valid  to 
assume  that  the  failure  is  of  the  "chance"  or  random  variety  and 
that  the  exponential  relation  applies.  This  subject  is  treated  in  many 
books  on  reliability.  (For  example,  see  pages  3-5  of  Reliability  Theory 
and  Practice,  by  Igor  Bazovsky.  Prentice-Hall,  1961.)  In  particular,  it 
should  be  noted  that  the  calibration  process  is  essentially  identical  to 
the  "overall"  process  referred  to  on  page  5. 


1.50,  or  2.00,  depending  on  which  zone  boundary  line 
is  being  calculated ) .  In  this  situation  the  reliability 
equation  R  =  e~bkt  applies  where  R  —  reliability, 
e  =  base  of  natural  logarithms,  k  is  the  failure  rate, 
and  t  is  a  variable  calibration  interval.  In  our  calcula- 
tions the  failure  rate  is  assumed  for  points  A  and  B. 
If  Ri  =  97.00  percent  at  A,  then  the  number  1.00  kt 
can  be  read  as  0.0305  from  tables  of  the  descending 
exponential  (as  in  the  NBS  Applied  Mathematics  Ser- 
ies 14).  Then  kt  is  multiplied  by  the  ratio  of  the 
new  calibration  interval  to  the  old,  and  the  new  value 
of  the  R,  estimate  is  rated  from  the  tables  using  expo- 
nent bkt.  In  figure  3,  b  =  1.50,  whence  bkt  —  0.04575 
and  Ri  =  95.53  percent;  this  is  independent  of  sam- 
ple size. 

Now  let  us  examine  the  second  step.  It  is  desired  to 
calculate  a  sample  size  such  that,  when  Ri  =  95.53  per- 
cent, the  probability  is  80  percent  that  Ra  is  92  per- 
cent or  greater.  This  involves  confidence  limits,  and 
for  these  calculations  the  booklet  Confidence  Limits 
for  Attributes  Data,  a  Lockheed  document,  has  been 
used.  (An  alternate  method  is  described  on  page 
373  of  Engineering  Statistics  by  Bowker  and  Lieber- 
man,  Prentice-Hall,  1959.  The  Lockheed  booklet  is 
used  because  the  calculations  are  much  simpler  and 
the  range  of  available  confidence  limits  is  wider).  A 
value  of  N  is  assumed,  the  number  of  failures  calculated 
as  N  (1  —  Ri).  and  the  80  percent  lower  limit  of  Ra 
found  in  the  tables.  If  Ra  is  over  92  percent,  the  as- 
sumed value  of  /V  was  too  high,  and  if  Ra  is  less  than 
92  percent,  it  is  too  low.  The  /V  and  Rt  values  thus 
found  fix  one  point  on  a  zone  boundary  line  in  figure 
2  and  the  process  is  repeated  for  enough  points  to  de- 
fine each  such  line  accurately.  As  might  be  deduced 
from  the  above  discussion,  point  C  of  figure  3  is  a 
function  of  point  B,  sample  size,  and  the  desired  con- 
fidence level.  When  sample  size  becomes  high  enough 
so  that  TV  (1  —  Ri)  is  greater  than  10,  the  Lockheed 
tables  are  no  longer  usable  and  some  other  method 
must  be  used.  When  sample  size  is  large  and  percent 
defective  small,  the  Poisson  distribution  may  be  used 
to  calculate  these  confidence  limits.  For  reference, 
see  paragraph  2.4.5,  page  438,  Quality  Control  and 
Industrial  Statistics,  Acheson  J.  Duncan,  Richard  D. 
Irwin,  Inc.,  1959,  or  many  other  books  on  mathemati- 
cal statistics.  Sample  calculations  of  two  boundary 
line  points  are  given  in  the  Appendix. 

The  statistical  techniques  used  to  calculate  boundary 
line  sample  sizes  when  there  are  no  defectives  in  the 
sample  (/?,=:100%)  are  necessarily  different  from 
those  above.  When  /?,  =  100  percent,  point  A  is  al- 
ways 100  percent  and  point  B  is  also  always  100  per- 
cent. However,  the  problem  can  be  solved  by  ap- 
proaching it  through  the  combination  of  sample  size 
and  the  magnitude  of  calibration  interval  change  per- 
mitted in  Zones  1,  2,  and  3  respectively.  Zone  3  will 
be  used  to  demonstrate  the  logic  involved  in  making 
the  calculations.  Zone  3  permits  the  calibration  in- 
terval to  be  doubled  and  when  the  calibration  interval 
is  doubled,  the  rate  of  data  input  is  approximately 
halved,  assuming  a  constant  population.  Therefore, 
in  the  case  of  Zone  3  and  Ri  =  100  percent,  we  need  to 
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answer  the  question,  "What  is  the  minimum  sample 
size  which,  when  divided  by  2,  provides  that  Ra  is  at 
least  92  percent  with  a  probability  of  80  percent?" 
The  Lockheed  tables  are  useful  in  making  this  deter- 
mination. Using  the  80  percent  lower  limit  table,  we 
see  that  a  sample  size  of  19  with  0  failures  has  an  80 
percent  lower  limit  of  only  91.88  percent  but  a  sample 
size  of  20  has  an  80  percent  lower  limit  of  92.27  per- 
cent. By  doubling  20,  we  obtain  40  as  the  minimum 
sample  for  Zone  3  when  /?j  =  100  percent.  Following 
the  same  logic,  sample  sizes  for  all  three  zones  are 
determined  as  follows: 


Zone 

A 

B 

C 

D 

1 

20 

0.75 

1.33 

27 

2 

20 

.67 

1.50 

30 

3 

20 

.50 

2.00 

40 

where  A  =  minimum  sample  at  which  R„  =  92  percent 
with  a  probability  of  80  percent  when 
Ri  =  100  percent. 
B  =  Amount  of  data  obtained  after  extension 
compared  to  that  obtained  prior  to  exten- 
sion. 

C  =  1/B  =  the  factor  by  which  A  must  be  mul- 
tiplied to  obtain  the  sample  size  which  per- 
mits extension. 

D  =  A  X  C  =  Minimum  sample  sizes  for  Zones 
1,  2,  and  3  when  /?,  =  100  percent. 

6.  Fixed  Interval  Through 
Engineering  Intuition 

The  use  of  the  "fixed  interval  through  data"  method 
is  limited  to  instrument  families  large  enough  so  that 
at  least  26  operative  instruments  are  serviced  in  some 
arbitrary  period  of  time  and  the  zones  of  figure  2  are 
calculated  on  the  assumption  that  the  data  are  a  sam- 
ple and  that  sampling  variations  have  been  taken  into 
consideration.  However,  there  are  many  instrument 
families  which  take  a  very  long  period  of  time  (2  to 
5  years  or  more)  before  27  operative  instruments  will 
be  serviced,  and  it  is  therefore  highly  desirable  to 
develop  other  criteria  applicable  to  instrument  families 
with  fewer  than  27  which  will  permit  calibration  in- 
terval extensions  for  them. 

To  proceed  along  these  lines  requires  dependence 
to  some  degree  on  considerations  other  than  statistical 
sampling  theory.  The  most  important  of  these  con- 
siderations are: 

1.  The  quantity  of  data  exceeds,  sometimes  by  a 
substantial  amount,  the  instrument  population,  so  that 


the  uncertainties  of  sampling  variation  incorporated 
into  the  other  method  are  diminished  by  a  great 
amount,  probably  an  order  of  magnitude. 

2.  The  period  of  time  over  which  the  data  are  accu- 
mulated exceeds,  sometimes  by  a  substantial  amount, 
a  full  calibration  interval  so  that  the  chance  that  an 
active  unit  might  not  be  included  in  the  data  is  quite 
minimal. 

3.  No  increase  in  calibration  interval  will  be  made 
if  there  are  any  out-of-tolerance  units  in  the  sample. 
Rn  is  calculated  in  the  normal  manner  and  must  be  at 
least  92  percent. 

4.  The  items  being  considered  for  interval  extension 
will  almost  always  have  some  comparable  items  on 
which  the  interval  was  extended,  using  data  for  which 
the  sampling  uncertainties  were  accounted  for. 

5.  Engineering  judgment  and  knowledge  concern- 
ing instrument  design,  function,  application,  and  usage 
is  a  legitimate  substitute  for  recorded  data,  even 
though  it  is  quite  difficult  to  qualify  that  judgment. 

6.  The  operational  risks  are  relatively  small  be- 
cause of  the  small  instrument  populations  involved. 

On  the  basis  of  these  factors,  table  I  identifies  per- 
missible interval  extensions  in  terms  of  both  the  abso- 
lute and  relative  amount  of  data  on  hand.  Examples 
of  changes  permissible  under  these  rules: 

Active  population  9,  calibration  interval 

3  months,  sample  size  23  — 
Category  I  —  Change  to  6  months 
Active  population  14,  calibration  interval 

6  months,  sample  size  19  — 
Category  II  —  Change  to  9  months 
Active  population  3,  calibration  interval 

4  months,  sample  size  8  — 
Category  III  —  Change  to  6  months 
Active  population  7,  calibration  interval 

9  months,  sample  size  11  — 
Category  IV  —  Change  to  12  months 
The  decision  matrix  of  table  1  has  some  character- 
istics which  distinguish  it  from  other  schemes  em- 
ploying intuitive  judgment: 

1.  All  the  intuition  is  applied  to  generation  of  a 
set  of  decision  rules,  thus  allowing  the  application  of 
judgment  to  specific  cases  to  be  precise  and  con- 
sistent. 

2.  Once  the  decision  matrix  is  made,  any  clerk, 
technician,  or  laboratory  personnel  can  apply  the  rules 
with  ease. 

3.  All  decisions  are  supported  by  a  strong,  well- 
defined  data  base. 


Table  1.  Permissible  extensions  of  calibration  intervals  with  5  to  26  bits  of  data,  fixed  interval  through  engineer- 


Category 

Sample 

Size 

Quantity  of  Data  Relative  to  Active 

Population 

Changes  Permissible,  months 

I 

16-26 

Minimum  of  double 

1-2,  2-1.  3-6, 

II 

16-26 

Equal  or  greater,  but  less  than 

double 

2-3.  3-4.  4-6,  6-9,  9-12,  12-18,  18-24 

III 

5-15 

Minimum  of  double 

2-3,  4-6,  6-9,  12-18 

IV 

5-15 

Equal  or  greater,  but  less  than 

double 

3-4.,  9_12,  18-24 

Note:  Changes  permissible  in  categories  II  and  IV  are  also  permissible  when  data  are  in  categories  I  and  III. 
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7.  Calibration  Interval  Decreases 

Theoretically,  the  logic  and  statistical  methods  for 
decreasing  calibration  intervals  and  thereby  bringing 
Ri  up  to  an  acceptable  level  should  be  the  same  as 
those  used  for  increasing  intervals.  As  a  practical 
matter,  this  does  not  work  at  all.  Assume  for  the 
moment  that  the  criteria  for  decreasing  intervals  are 
made  comparable  to  those  for  increasing  intervals: 

•  Ri  must  be  less  than  85  percent. 

•  At  the  new  interval,  the  probability  is  80  percent 
that  Ra  is  at  least  92  percent. 
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Figure  4.  Chart  for  determination  of  calibration 
interval  decreases. 


Note  that  the  second  criterion  is  identical  to  that  for 
increasing  intervals.  Using  the  same  calculation  meth- 
ods as  previously,  zone  boundary  lines  were  calculated 
and  are  shown  in  figure  4.  The  differences  between 
figure  2  and  figure  4  are  very  startling: 

•  The  "no  change"  zone  is  very  small. 

•  The  zones  are  reversed  in  that  the  most  severe 
change  zone  (1)  is  next  to  the  no  change  zone  (0) 
and  the  least  severe  change  zone  (4)  is  farthest  re- 
moved from  the  no  change  zone  (0). 

•  Except  for  the  no  change  zone,  the  smaller  the 
sample,  the  more  severe  the  change. 

•  There  are  no  zones  at  all  for  reducing  intervals 
by  25  percent  (12  to  9  months,  for  example),  33  per- 
cent (9  to  6  months,  for  example),  or  50  percent  (12 
to  6  months,  for  example). 


The  second  and  third  points  are  explained  by  the 
fact  that:  the  smaller  the  sample,  the  larger  the  un- 
certainty; and  the  larger  the  uncertainty,  the  more  se- 
vere the  necessary  action  to  be  sure  that  desired  re- 
sults are  forthcoming.  The  last  point  shows  that  there 
is  no  value  of  R,  below  85  percent  such  that  a  25,  33, 
or  50  percent  decrease  in  interval  will  make  Ra  —  92 
percent  with  any  reasonable  degree  of  certainty. 

These  conditions,  therefore,  show  rather  clearly  the 
severe  limitations  of  attempting  to  improve  R,  by  de- 
creasing the  interval  and  the  prudence  of  exploring 
other,  more  fruitful  corrective  actions. 

8.  Procedure  and  Documentation 

When  a  calibration  interval  change  is  to  be  made, 
a  Reliability  Bulletin  is  initiated  by  the  Measurement 
Standards  Laboratory  (MSL)  Reliability  Engineer, 
who  provides  the  required  data,  as  shown  in  table  2. 
The  bulletin  is  signed  by  the  MSL  Manager  and  ap- 
proved by  the  cognizant  Navy  and  Air  Force  Cali- 
bration Specialists.  It  thus  provides  the  vehicle  for 
generation  and  approval  of  a  change  and  also  pro- 
vides permanent  documentation  for  the  change. 

9.  Limitations  on  Calibration 
Interval  Extensions 

There  are  several  situations  and  conditions  where 
it  is  prudent,  at  least  in  the  initial  phases  of  applica- 
tion, to  impose  specific  arbitrary  limitations  on  inter- 
val extensions,  even  though  the  data  might  permit 
greater  or  more  frequent  extensions: 

1.  The  maximum  individual  extension  is  3  months 
for  all  instruments  except  those  with  12  or  18  month 
intervals,  where  an  increase  of  6  months  is  permitted. 

2.  The  data  used  to  support  a  change  shall  cover  a 
period  equal  to  at  least  one-half  the  current  interval, 
regardless  of  how  much  might  be  available  over  a 
shorter  period. 

3.  After  an  interval  extension  has  been  made,  no 
further  extension  can  be  considered  until  a  period  of 
time  equal  to  the  sum  of  the  old  and  new  intervals  has 
elapsed,  and  none  of  the  data  used  to  consider  a  sec- 
ond extension  shall  be  older  than  the  new  calibration 
interval.  If  a  change  from  4  to  6  months  is  made,  for 
example,  no  further  extension  can  be  considered  for 


Table  2.  A  typical  reliability  bulletin 


NOTIFICATION  OF  RECALL  PERIOD  CHANGE 


Manufacturer/Model         —       Electronic  Measurement  TO  Series  Power  Supplies 

Station  Code  —  LEMK 

Change  to  be  Made  —       09  to  12  months 

Effectivity  Date  —      2  March  1970 

Quantity  Involved  —  27 

Data  Supporting  Change: 
For  the  13  months  ending  January  1970  — 
Total  calibrated    —  68 
Operative  —  58,  R0  =  97.1% 

In-tolerance        —  57,  Rc  =  98.2% 
A  =  7     D  =  1     P  =  27 

Sample  size  58,  Rc  =  98.2  percent  falls  into  Zone  3  which  permits  extension  of  the  recall  period  from  09  to  12  months. 
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10  months  and  the  data  for  considering  an  extension 
at  that  time  can  be  no  more  than  6  months  old.  This 
limitation  accomplishes  two  things:  (a)  it  allows  the 
data  bank  to  be  purged  of  all  data  from  the  old  in- 
terval period,  and  I  b )  it  allows  for  a  full  cycle  of  the 
population  to  be  processed  and  therefore  assures  that 
any  subsequent  consideration  will  be  based  on  data 
from  the  entire  population. 

4.  The  calibration  interval  of  an  instrument  pow- 
ered by  a  battery  with  a  limited  life,  an  instrument 
needing  regular  mechanical  attention,  or  one  with 
similar  functional  time-dependent  limitations,  will  be 
governed  by  those  considerations  and  not  by  R  JRU- 

5.  Calibration  interval  extensions  shall  be  made 
only  four  times  per  year  on  the  first  Monday  of 
March.  June,  September,  and  December.  These  times 
coincide  with  the  new  printing  of  preprinted  Service 
Reports,  with  calibration  interval  being  part  of  the 
preprinted  information. 

10.  Results 

The  family  size  for  more  than  half  the  equipment 
reaching  MSL  is  between  1  and  9  instruments;  other 
percentages  are  15  percent  for  10  to  19  items;  18  per- 
cent for  20  to  49.  8  percent  for  50  to  99,  4  percent  for 
100  to  199,  and  two  percent  for  200  or  more.  The 
first  changes  made  by  the  two-method  system  de- 
scribed were  effective  1  December  1968.  Including 
those  which  were  effective  on  2  March  1970,  results 
have  been  as  follows: 

•  Total  interval  changes — 495. 

•  Total  instruments  involved — 34,200. 

•  Maintenance  of  88-92  percent  in-tolerance  relia- 
bility, as  shown  by  figure  5. 

•  Steady  reduction  in  total  scheduled  calibrations 
per  unit  per  year  from  2.35  to  2.06,  or  12.3  percent. 


•  Steady  reduction  in  total  scheduled  calibrations 
per  year — 16.6  percent  from  119,600  to  99,600  or 
16.6  percent.  The  difference  between  16.6  percent 
and  12.3  percent  is  accounted  for  by  a  4.3  percent 
drop  in  active  instrument  population. 

•  Exponential  reduction  in  annual  technician  hours. 
The  reduction  is  expected  to  reach  24,100  hours  by 
August  1971.  Note  that  a  full  interval  must  elapse 
before  the  reduction  associated  with  a  specific  change 
begins  to  be  realized. 

•  Major  changes  in  distribution  of  calibration  in- 
tervals, as  shown  in  figure  6,  where  each  stepped 
column  covers  seven  quarters.  The  pattern  of  changes 
is  quite  clear.  Early  in  the  program,  emphasis  was 
on  changes  from  3  to  4,  3  to  6,  and  4  to  6  months, 
since  39  percent  of  the  total  population  was  on  three- 
month  intervals  and  the  yield  in  hours  is  high.  Later 
the  emphasis  shifted  to  changes  from  6  to  9  and  9  to 
12  months,  and  finally  a  substantial  quantity  from  12 
to  18  months. 

With  the  passage  of  time,  many  more  extensions 
will  be  made  and  a  considerable  number  of  second 
and  possibly  even  third  extensions.  It  is  estimated 
that  by  the  end  of  1972  the  total  saving  in  annual 
technician  time  will  be  on  the  order  of  45,000  to 
50,000  hours. 


The  writer  wishes  to  acknowledge  the  large  con- 
tribution of  Messrs.  D.  B.  Schneider,  Research  Spe- 
cialist, and  C.  W.  Gebhardt.  Reliability  Engineer,  the 
Lockheed  Measurement  Standards  Laboratories,  to 
the  theory  and  development  of  the  statistical  aspects 
of  the  system.  Also  instrumental  in  the  overall  for- 
mulation of  these  methods  was  Mr.  M.  L.  Brink  of 
the  Navy  Special  Projects  Office. 
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Figure  5.  In-tolerance  reliability,  1969  and  1970. 
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Figure  6.  Quarterly  changes  in  distribution  of  calibration  intervals. 


11.  Appendix:  Sample  Calculations  of  Boundary  Line  Points 


The  calculations  below  show  determination  of  two 
points  on  the  2.00  t  boundary  line  (between  Zone  2 
and  Zone  3)  in  figure  2. 

Point  1 

Ri  =  0.9800  at  1.00  t 

Ri  =  0.9604  at  2.00  t,  p  =  1  -  0.9604  =  0.0396 
N  =  60    Np  =  2.376    Ra  =  0.9225 

(assumed  N  too  large) 
N  =  50    Np  =  1.980    Ra  =  0.9169 

(assumed  N  too  small) 
N  =  55    /Vp  =  2.178    Ra  =  0.9199 

(assumed  N  slightly  small) 
N  =  56    /Vp  =  2.218    Ra  =  0.9205 

Therefore,  /V  =  56,  Ri  —  98  percent  are  the  coordi- 
nates. 

Point  2  (using  Molina's  Tables  of  Poisson's  Exponen- 
tial Binomial  Limit,  D.  Van  Nostrand,  1947) 
Ri  =  0.9650  at  1.00  t 

Ri  =  0.9312  at  2.00  t,  p  =  1  -  0.9312  =  0.0688 
N  =  726.7 2    Np  =  50    Np  (80%)  =  56.41 

Ra  =  1  -  56.41/726.7  =  0.9224 
N  =  581.4    Np  =  40   Np  (80% )  =  45.78 

Ra  =  1  -  45.78/581.4  =  0.9213 


yV  =  436.0    /Vp  =  30    /Vp  ( 80% )  =  35.06 

Ra  =  l-  35.06/436.0  =  0.9196 
N  =  465.1    Np  =  32    Np  (80% )  =  37.22 

Ra  =  l~  37.22/465.1  =  0.9200 

Therefore,  N  =  465,  R,  =  0.965  =  96.5  percent  are 
the  coordinates. 


Zone  0 — No  change 


Zone  2— 1.50f 
2  to  3  months 
4  to  6  months 
6  to  9  months 
12  to  18  months 


Zone  1 — l.S3t 
3  to  4  months 
9  to  12  months 
18  to  24  months 

Zone  3— 2.00* 

1  to  2  months 

2  to  4  months 

3  to  6  months 


2  Non-integral  sample  sizes  used  so  that  Np  can  be  integral  to 
simplify  interpolation  in  the  tables. 


Example  of  use:  sample  size  =  75;  in-tolerance  =  73;  Ri  — 
73/75  =  97.3  percent ;  t  =  6  months. 

Sample  size  =  75  and  Ri  =  97.3  percent  lies  in  Zone  2,  per- 
mitting extension  to  9  months. 

Zone  0 — No  change;  Zone  1 — Reduce  interval  by  more  than 
75  percent;  Zone  2 — Reduce  interval  by  75  percent;  Zone  3 — 
reduce  interval  by  66  percent. 

Point  A  =  Rj  at  1.00  t;  point  B  =  best  estimate  of  Ri  at  1.50  t; 
point  C  =  Rn  is  equal  to  or  greater  than  92  percent  over  80  per- 
cent of  the  time 
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OPTIMUM  CYCLES  DETERMINED  WITH  CONFIDENCE 

P.  A.  Seamans 

Electronics  Laboratory,  General  Electric  Company,  Syracuse,  N.Y.  13201 

The  calibration  history  of  a  family  of  instruments  provides  data  on  how  often  the  instruments  were 
recertified  and  whether  each  was  in  calibration  when  recertified.  These  data  permit  the  calculation 
of  the  optimum  calibration  interval  for  a  desired  level  of  confidence  in  the  instrument  readings. 
Three  case  histories  (oscilloscope,  RX  meter,  VTVM)  show  this  method  is  effective.  The  histories  of 
a  family  of  oscilloscopes  over  a  thirteen-year  period  show  that  calibration  cost  tends  to  be  roughly 
constant  with  time  and  thus  does  not  seem  to  follow  the  "bathtub  curve."  All  instruments  inves- 
tigated seem  to  have  equal  likelihood  of  failure  in  all  intervals,  and  thus  the  bathtub  curve  does  not 
seem  to  hold  in  general. 

Key  words:  Calibration  histories;  equal  likelihood;  oscilloscopes;  RX  meters;  vacuum-tube  voltmeters. 


1.  Introduction 

For  the  past  several  years  we  have  been  using  a 
technique  for  determining  the  optimum  length  of  time 
between  successive  calibrations  of  different  kinds  of 
instruments.  The  technique  uses  data  from  the  cali- 
bration history  of  a  given  family  of  instruments  and 
allows  a  balance  to  be  struck  between  the  cost  of 
errors  due  to  possibly  faulty  equipment  and  the  cost 
of  calibrations.  The  technique  gives  a  numerical 
measure  of  the  level  of  confidence  the  instrument  user 
can  have  in  the  readings  his  instrument  gives.  The 
technique  has  been  proven  effective  for  a  wide  variety 
of  scientific  equipment  used  in  the  electronics  indus- 
try, and  it  is  probably  suitable  for  most  instruments. 


2.  Collecting  Instrument  Data 

We  have  established  in  the  Instrumentation  and 
Measurements  Subsection  at  the  General  Electric  Elec- 
tronics Laboratory  a  computer-based  system  of  keep- 
ing records  on  the  calibration  and  maintenance  of 
many  kinds  of  instruments.1  This  system  keeps  histo- 
ries of  over  10,000  instruments  with  about  30  man- 
hours  a  week  of  effort,  largely  of  a  clerical  nature. 
Data  processing  cards  (DPC)  that  record  pertinent 
information  about  the  calibration  of  an  instrument 


1  This  computer-based  system  is  fully  described  in  a  report,  Instru- 
ment Calibration  Records:  Establishment  of  a  High-Confidence  Data 
Bank.  Interested  readers  should  write  Technical  Information,  Gen- 
eral Electric  Company,  Electronics  Laboratory,  Syracuse,  N.Y.  13201, 
and  request  TIS  Report  R69ELS-115. 
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Figure  1.    Flow-chart  of  computer-based  system  for  keeping 
instrument  calibration  records. 
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form  the  input  to  the  computer  ( fig.  1 )  and  "reports" 
of  various  kinds  form  an  output  which  permits  esti- 
mation of  optimum  calibration  intervals. 

One  of  the  computer  reports  is  a  "history  report," 
consisting  of  the  calibration  history  of  a  given  family 
of  instruments  and  useful  in  calculating  the  optimum 
intervals  between  calibrations.  Figure  2  shows  a 
sample  history  report  for  a  certain  kind  of  oscillo- 
scope plug-in.  It  includes  an  instrument  code  that 
corresponds  to  the  family  of  instruments;  the  name 
of  the  manufacturer  and  his  model  designation;  the 
identification  serial  number  assigned  to  each  different 
I  instrument;  and  a  work  tag  number,  there  being  one 
work  tag  for  each  job  that  is  performed  in  our  cali- 
bration area.  Other  entries  indicate  the  date  the  cali- 
bration was  performed,  the  technician  who  performed 
the  calibration,  the  nature  of  the  trouble,  and  the  cost 
of  calibration  or  repair.  But  of  major  interest  to  us 
in  determining  the  optimum  interval  between  calibra- 
tions are  the  date  entries  and  the  "status"  entries, 
which  indicate  (among  other  things)  whether  an  in- 
strument was  found  to  be  in  calibration  when  it  was 
checked.  These  two  entries  are  used  to  derive  the 
actual  number  of  calibrations  per  year  and  the  actual 
number  of  calibrations  per  defect,  which  in  turn  are 
used  to  calculate  the  optimum  number  of  calibrations 
per  year. 

After  an  instrument  has  been  judged  in  or  out  of 
calibration,  it  is  either  recalibrated  or  given  "pre- 
ventive maintenance,"  whichever  is  appropriate.  (Pre- 
ventive maintenance  might  involve  bringing  the  instru- 
ment's performance  closer  to  nominal  values  even 
though  it  was  within  manufacturer's  specifications; 
replacing  a  weak  battery  or  tube  to  avoid  failure 
before  the  next  recertification  date;  or  cleaning  switch 
contacts  or  wiper  arm  contacts  to  permit  intermittent 
readings  during  the  time  until  the  next  calibration.) 

Closely  associated  with  the  in-calibration  or  out-of- 
calibration  status  of  an  instrument  is  something  we 
call  "failure" — it  occurs  when  the  calibrating  per- 
sonnel determine  that  erroneous  readings  could  have 
been  made  without  the  user's  knowledge.  An  example 
might  be  a  pointer  that  is  bent,  though  not  enough 
to  be  noticeable  to  the  inattentive  eye;  or  there  might 
be  friction  or  jewel  roll  present.  "Failure"  would 
have  occurred  if  the  instrument  was  out  of  balance 
and  was  used  in  several  planes.  A  more  subtle  exam- 
ple might  be  a  defect  in  an  oscilloscope  power  supply 
that  causes  the  rise-time  trace  to  be  incorrect,  a 
"failure"  that  may  be  more  serious  than  if  the  oscillo- 
scope did  not  function  at  all.  In  fact,  if  the  power 
supply  had  failed  (in  the  conventional  sense),  then 
in  accordance  with  our  definition  it  would  not  be  a 
"failure." 

3.  Calculating  the  Optimum 
Calibration  Interval 

The  data  found  in  the  history  report  (fig.  2)  en- 
able us  to  calculate  the  actual  number  of  calibrations 
per  year,  (C/t),  for  a  given  family  of  instruments, 
where  C  is  the  number  of  calibrations  performed  over 
a  period  of  t  years;  and  the  actual  number  of  calibra- 


tions per  defect,  (C/D),  where  D  is  the  number  of 
defects,  or  instances  of  an  instrument's  being  out  of 
calibration.  Then  it  can  be  shown  that  the  optimum 
number  of  calibrations  per  year  can  be  expressed: 

(CA)opt  =  (C/t)  '^°pt 

where  the  term  (C/D)opl,  the  optimum  number  of 
calibrations  per  defect,  is  found  by  entering  a  Poisson 
distribution  table  for  a  given  confidence  level. 

An  example  will  indicate  the  utility  of  this  expres- 
sion: Suppose  that,  for  a  particular  family  of  instru- 
ments, calibration  takes  place  four  times  a  year  and 
that  to  date  a  total  of  158  such  calibrations  have  been 
made,  including  46  "failures."  Then  we  have  (C/t) 
=  4,  and  (C/D)  =  (158/46)  =  3.43.  If  we  wish  90 
percent  of  the  instruments  returning  for  calibration 
to  be  within  manufacturer's  specifications,  then  we 
find  from  the  short  Poisson  table  (table  1)  that 
(C/D)cpl  =  3.9,  and  the  above  expression  becomes: 

(C/«)w  =4(||) 

=  4.55  calibrations  per  year 

(calibration  every  IIV2  weeks) 


Table  1.  Poisson  distribution 
for  at  least  one  failure  occurring  in  interval 


Confidence  Level 

(C/D)opt 
(Calibrations  Per  Defect) 

% 

99 

6.7 

95 

4.7 

90 

3.9 

85 

3.4 

4.  Evaluation  of  the  Technique 

Our  experience  with  this  technique  proves  that  it 
works  and  is  applicable  to  test  equipment.  To  dem- 
onstrate that  we  can  predict  optimum  calibration  in- 
tervals with  confidence,  I  cite  three  examples: 

Generators  prior  to  1968  had  been  calibrated  every 
26  weeks.  When  we  first  applied  our  method  of  find- 
ing optimum  calibration  intervals,  we  found  that  in 
order  to  maintain  a  90  percent  confidence  level,  the 
cycle  should  be  shortened  to  15  weeks.  When  we 
had  collected  a  year's  data  at  the  new  interval,  we 
computed  the  optimum  value  again  and  found  it  to  be 
18  weeks.  (The  reason  for  the  three-week  discrep- 
ancy is  unclear,  but  the  important  thing  to  note  is 
that  our  first  attempt  to  adjust  the  optimum  value 
was  in  the  correct  direction.) 

Perhaps  a  better  example  for  illustrating  the  accu- 
racy of  this  technique  would  be  our  experience  with 
RX  meters.  As  in  the  case  of  the  generators,  the 
calibration  interval  for  these  instruments  had  been  26 
weeks.  Our  computations  showed  that  the  cycle  should 
be  shortened  to  15.8  weeks.  We  thus  shortened  the 
cycle  to  15  weeks;  at  the  end  of  a  year  we  checked  the 
results  and  found  the  optimum  to  be  14.7  weeks — very 
close  to  the  originally  calculated  value. 
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A  third  example,  involving  a  lengthening  rather 
than  a  shortening  of  the  calibration  interval,  involves 
several  families  of  VTVM's.  One  of  these  families 
was  routinely  calibrated  every  13  weeks,  but  the  others 
were  calibrated  every  four  weeks.  After  we  studied 
the  failure  rate  of  these  instruments,  we  found  we 
could  lengthen  the  calibration  intervals  5,  7,  or  10 
weeks  for  those  that  had  been  calibrated  every  four 
weeks;  and  the  thirteen-week  instruments  were  now 
calibrated  every  26  weeks.  A  subsequent  study  after 
a  year's  operation  showed  these  revised  calibration 
intervals  still  to  be  correct. 

5.  Discussion 

Many  readers  are  perhaps  acquainted  with  the 
"bathtub  curve"  that  is  said  to  reflect  the  life  and 
death  of  instruments  (fig.  3).  It  is  my  opinion  that 
this  curve  is  not  entirely  applicable  to  the  broad  spec- 
trum of  test  equipment,  although  perhaps  some  me- 
chanical systems  follow  it.  If  test  equipment  obeyed 
the  bathtub  law,  the  age  of  a  piece  of  equipment  would 
be  the  determining  factor  of  the  calibration  interval. 
Instruments  would  have  to  be  segregated  by  age,  and 
in  a  large  calibration  area  the  economics  of  instru- 
ment maintenance  would  be  difficult.  It  would  then 
be  possible  that  instruments  purchased  this  year  would 
have  a  different  calibration  cycle  than  the  same  type 
purchased  a  year  or  two  ago.  But  the  technique  I 
have  described  for  determining  the  optimum  calibra- 
tion interval  assumes  a  Poisson  distribution,  which  is 
used  when  the  same  number  of  failures  are  expected 
in  each  interval.    To  me  this  suggests  that  test  equip- 
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Figure  3.   The  well-known  "bathtub  curve"  that 
is  said  to  reflect  the  useful  life  of  instruments. 


ment  failures  might  be  a  linear  function  of  use,  de- 
pending on  environment  and  care,  rather  than  on  age 
as  long  as  the  age  is  reasonable. 

I  have  recently  completed  a  study  of  a  well-known 
manufacturer's  oscilloscope.  I  collected  data  on  29 
instruments,  including  purchase  date  and  the  average 
cost  of  calibration  for  four  years  (1966-69).  A  total 
of  252  calibrations  was  included  in  the  study  of  these 
instruments,  some  of  which  were  purchased  as  long 
ago  as  1957.  A  plot  of  these  data  is  scattered  about 
a  horizontal  line  (fig.  4)  rather  than  the  bathtub 
mentioned  above.  The  cost  of  calibrating  instruments 
12  years  old  was  not  much  different  from  that  of 
equipment  purchased  quite  recently.  The  anomalous 
value  at  10  years  (*)  results  from  a  modification  which 
involved  expensive  repairs. 
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Figure  4.  The  1966-69  cost  of  calibrating  29  oscil- 
loscopes aged  1  to  12  years  scatters  about  a  hori- 
zontal line  and  does  not  seem  to  follow  the  bathtub 
curve. 

The  procedure  described  may  bring  about  a  possi- 
ble savings  by  the  mere  fact  that  an  analysis  of  the 
history  of  instruments  may  permit  the  elimination  of 
"dogs."  Furthermore,  some  cycles  can  perhaps  be 
lengthened,  resulting  in  further  savings.  The  method 
is,  of  course,  of  little  use  if  some  sort  of  systematically 
scheduled  calibration  system  is  not  in  force.  In  re- 
viewing any  family  of  instruments,  I  include  only 
those  instruments  that  have  been  recalibrated  at  least 
half  as  often  as  the  optimum  calibration  interval  would 
require. 
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Metrology  work  involves  the  conflicting  and  diametrically  opposed  problems  of  control  versus 
breakthrough.  Controls  are  vital  to  assure  accuracy  and  reliability.  However,  breakthroughs  are 
necessary  to  provide  improved  accuracies,  ranges,  and  types  of  measurements.  The  dichotomy  of  this 
situation  comes  principally  from  the  differences  in  attitudes  involved.  Ideally,  control  and  break- 
through should  be  carried  out  by  two  different  types  of  people  because  of  these  differences. 

Breakthrough  techniques,  coupled  with  proper  attitudes  and  a  systematic  method  for  establishing 
objectives  and  evaluating  alternatives,  provide  a  powerful  set  of  new  tools  for  improvement.  Examples 
are  given  showing  the  possibilities  for  applying  these  tools  to  reduce  costs  and  provide  needed 
measurement  and  calibration  services. 

Key  words:  Managerial  breakthrough  techniques;  personnel  selection  for  metrology;  steady  control 
versus  sudden  change. 


1.  Introduction 

This  is  a  paper  about  problem  solving.  The  type 
of  problem  which  will  be  covered  is  breakthrough, 
where  breakthrough  is  defined  as  a  process  by  which 
improvements  of  a  major  nature  are  made  rapidly. 
Breakthrough  obviously  relies  heavily  on  creativity 
and  innovation.  However,  it  is  more  than  this  be- 
cause of  the  rapidity  of  the  change  involved;  break- 
through is  a  dynamic  word  which  denotes  a  sudden 
or  step-function  type  of  change. 

This  paper  stresses  the  achievement  of  breakthrough 
on  a  systematic  basis.  This  is  possible  because  break- 
through, like  many  other  types  of  problem  solving,  has 
become  a  science  which  can  be  performed  on  a 
rational,  step-by-step  basis.  Evidence  of  this  is  found 
in  the  literature  on  the  subject  of  breakthrough  itself, 
as  well  as  on  closely  related  subjects  such  as  creativity, 
innovation,  and  various  processes  for  analyzing  situa- 
tions and  making  decisions.  The  information  in  this 
paper  is  based  on  these  key  points  which  have  been 
condensed  into  an  integrated  and  relatively  simple 
approach  to  breakthrough,  with  particular  attention 
to  its  application  in  metrology  work. 

This  paper  has  several  purposes,  as  follows: 

(a)  To  explain  why  breakthrough  techniques  are 
desirable  and  feasible  for  more  extensive  use  in 
metrology  work. 

(b)  To  give  information  about  how  breakthrough 


can  be  achieved  and  to  provide  leads  to  sources  of 
information  about  more  detailed  aspects  of  break- 
through. 

(c)  To  suggest  some  metrology  areas  which  need 
breakthroughs  and  also  some  possible  ways  of  attain- 
ing such  breakthroughs. 

2.  Characteristics  of  Metrology  Problems 
Requiring  Breakthrough 

Breakthrough  techniques,  coupled  with  success- 
oriented  attitudes  and  a  systematic  method  of  estab- 
lishing objectives  and  evaluating  alternatives,  pro- 
vides a  powerful  set  of  new  tools  for  solving  improve- 
ment type  problems.  Today  there  are  increasing 
pressures  to  come  up  with  faster  and  better  solutions 
to  such  problems.  These  pressures  arise  from  the 
following  characteristics  which  many  metrology  prob- 
lems have: 

•  Metrology  work  involves  the  conflicting  and  di- 
ametrically opposed  problems  of  control  versus 
breakthrough.  Controls  are  vital  to  assure  accuracy 
and  reliability  of  metrology  work.  However,  break- 
throughs are  necessary  to  provide  improved  accura- 
cies, ranges,  and  types  of  measurements.  The  dichot- 
omy of  this  situation  comes  from  the  differences  in 
attitudes  of  the  people  involved.  Ideally,  control  and 
breakthrough  should  be  carried  out  by  two  different 
sets  of  people  because  of  these  differences. 
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•  Metrology  problems  are  complex;  they  have 
many  variables. 

•  Metrology  problems  are  generic  in  the  sense  that 
they  apply  in  about  the  same  way  to  many  different 
organizations;  i.e.,  they  cross  organizational  lines  and 
therefore  appear  not  to  be  the  responsibility  of  any 
one  organization. 

•  Adequate  attention  and  funding  are  difficult  to 
justify  because  the  problems  are  principally  of  a 
support  nature.  This  means  that  they  are  far  from 
the  basic  problem  sources — i.e.,  the  exotic  techno- 
logical requirements  of  space  work,  military  work,  etc. 

3.  Breakthrough  Versus  Control 

One  of  the  better  explanations  of  breakthrough  is 
that  given  in  Dr.  J.  M.  Juran's  Book,  Managerial 
Breakthrough  [l].1  This  explanation  is  particularly 
good  because  breakthrough  is  described  by  comparing 
it  with  its  opposite  function — control.  For  example, 
Dr.  Juran  makes  the  statement  that  "All  managerial 
activity  is  directed  at  either  breakthrough  or  control. 
Managers  are  busy  doing  both  of  these  things  and 
nothing  else."  Dr.  Juran  also  describes  breakthrough 
as  the  creation  of  good  or  necessary  changes,  while 
control  involves  the  prevention  of  undesirable  changes. 
Obviously,  both  types  of  activities  are  necessary  in  an 
organization,  although  the  type  and  amount  of  each 
will  vary  from  time  to  time  and  from  one  organiza- 
tion to  another. 

The  differences  in  the  results  expected  from  control 
and  from  breakthrough  cause  radical  differences  in 
the  methods  of  obtaining  each  of  them.  This  is  par- 
ticularly true  in  the  use  of  people,  for  either  break- 
through work  or  for  control  work.     The  selection 
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Figure  1.    Coordinate  analysis  of  attitudes  toward 
control  versus  improvement. 

1  Figures  in  brackets  indicate  the  literature  references  at  the  end 
of  this  paper. 


of  people  for  either  breakthrough  or  control  work 
should  take  into  account  their  education,  their  work 
experience,  their  motivation,  and  their  attitudes. 
However,  people  are  rarely  100  percent  control- 
oriented.  Instead,  there  are  many  possible  combina- 
tions, as  are  shown  in  the  coordinate  analysis  chart 
of  figure  1. 

4.  Breakthrough  Concepts 

People  are  basically  goal-oriented;  that  is,  they 
continually  set  goals  for  themselves,  work  to  reach 
such  goals,  and,  when  each  goal  is  reached,  select  a 
new  goal  to  work  towards.  Goals  for  achieving  con- 
trol are  usually  given  highest  priority,  particularly 
for  the  solution  of  crisis  or  "firefighting"  types  of 
problems.  However,  when  there  is  is  additional  time 
available,  it  is  usually  spent  on  improvement-type 
goals.  Breakthrough,  in  its  most  general  sense,  is  the 
technique  used  to  attain  improvement-type  goals.  In 
a  more  restricted  sense,  breakthrough  is  the  technique 
used  to  attain  improvement  goals  in  a  relatively  short 
period  of  time — short,  that  is,  with  respect  to  the  time 
such  improvement  changes  might  normally  be  expected 
to  take  on  a  strictly  evolutionary  basis. 

Some  of  the  important  characteristics  of  break- 
through are  as  follows: 

•  A  systematic  approach  is  usually  the  best  for 
achieving  success  in  breakthrough  work.  The  syste- 
matic approach  increases  the  probability  of  reaching 
the  breakthrough  objectives  and  cuts  the  time  require- 
ments compared  with  a  nonsystematic  approach. 

•  For  breakthrough  to  get  started,  there  must  first 
be  a  promoter  or  advocate — i.e.,  someone  who  feels 
that  improvement  is  not  only  necessary  but  that  it 
also  is  feasible. 

•  Breakthrough  requires  the  following  types  of 
leadership  qualities: 

— Constructive  discontent  with  the  status  quo 

— Willingness  to  take  risks 

— Good  sense  of  timing 

— Aggressiveness 

— Decisiveness 

— Confidence. 

•  The  amount  and  kinds  of  breakthrough  which 
can  be  achieved  are  directly  related  to  the  favorability 
of  the  climate  in  the  organization.  A  poor  organiza- 
tional climate  will  mean  that  breakthrough  is  likely  to 
be  either  impossible  or  at  least  stunted  and  mediocre. 

— The  most  important  climate  factor  in  an  or- 
ganization is  the  attitudes  towards  breakthrough  of 
top  management  and  of  the  immediate  boss  of  each 
person  involved  in  breakthrough. 

—The  second  most  important  climate  factor  in 
an  organization  is  the  attitudes  towards  breakthrough 
of  a  person's  peers  as  well  as  the  attitudes  of  the 
people  working  for  him  on  any  aspect  of  a  break- 
through task. 

5.  Steps  for  Achieving  Breakthrough 

The  nature  of  breakthrough-type  problem  solving 
is  shown  in  the  block  diagram  of  figure  2.  Reading 
horizontally  across  the  top  of  the  diagram,  we  find  a 
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Figure  2.  Steps  in  breakthrough-type  problem  solving. 


simplified  process  for  solving  problems  in  the  four 
steps — recognize  problem,  devise  solution,  implement 
solution,  and  follow  up  and  control  to  get  the  desired 
results.  However,  under  the  "devise  solution"  step 
the  diagram  also  shows  an  additional  series  of  steps — 
i.e.,  clarify  objectives;  generate  alternative  possible 
solutions;  analyze  constraints  in  terms  of  adverse  con- 
sequences, resource  limitations,  and  people-generated 
stalls,  objections,  and  roadblocks;  make  decision 
based  on  comparison  of  alternatives  with  constraints; 
and  analyze  potential  problems.  Ideally,  all  that  should 
be  necessary  to  do  is  go  through  the  steps  shown 
here  and  you  will  automatically  be  able  to  achieve 
breakthroughs  in  any  problem  area.  However,  unless 
care  is  taken  the  results  will  be  far  from  ideal — usually 
because  of  (a)  inadequate  attention  to  the  people- 
problems  involved  in  achieving  breakthrough,  or  (b) 
the  difficulties  imposed  by  state-of-the-art  technology 
limitations.  The  approaches  discussed  here  should 
be  able  to  help  avoid  or  overcome  these  limitations 
and  obstacles  to  a  considerable  extent. 

The  basic  approach  to  obtain  breakthrough  should 
involve  the  following  key  elements: 

— Using  a  systematic  approach  based  on  the  steps 
shown  in  the  block  diagram  of  figure  2. 

— Paying  attention  to  more  than  just  getting  the 
basic  ideas  on  how  to  achieve  the  desired  type  of 
breakthrough.  A  "total"  or  systems  approach  must 
be  used;  this  involves  solving  all  of  the  subproblems 
necessary  to  convert  the  basic  idea  into  a  realistic 
service,  method,  procedure,  or  a  working  piece  of 
hardware  or  software. 

— Setting  objectives  or  goals.  To  help  these  goals 
come  true,  they  must  be  broken  into  sub-goals,  with 
plans  for  achieving  each  of  the  sub-goals.  The  goals 
and  sub-goals  must  be  put  in  writing  in  order  to  clar- 
ify them  and  to  make  them  understandable  to  people 
who  must  work  to  achieve  them.  These  same  people 
must  identify  with  the  goals  by  seeing  the  relationship 
of  the  breakthrough  goals  and  sub-goals  with  their 
own  personal  goals  and  by  using  visualization  tech- 
niques to  keep  the  goals  and  sub-goals  clearly  in  mind 
while  working  towards  them.  The  concepts  of  selec- 
tive perception  and  positive  expectation  come  into  play 
when  goals  are  treated  in  this  way.    Selective  per- 


ception helps  people  to  become  more  sensitive  to 
anything  which  will  aid  them  in  achieving  goals  which 
they  understand  and  have  identified  with  [2] . 

— Generating  breakthrough-oriented  attitudes  in  the 
people  assigned  to  work  on  the  breakthrough  project. 
This  includes  inducing  in  each  person  a  positive  men- 
tal attitude  and  a  self-image  which  is  directed  towards 
successful  completion  of  the  breakthrough  project. 

— Use  of  various  creativity  and  innovation  tech- 
niques to  achieve  a  broad  spectrum  of  alternative 
means  of  solving  the  main  breakthrough  problem  and 
its  subproblems. 

— Clarifying  the  objectives  through  the  use  of  tech- 
niques such  as  situation  analysis,  critical  factor  analy- 
sis, and  function  analysis.  This  helps  avoid  the  effects 
of  finding  the  right  solution  for  the  wrong  objectives. 

— Including  in  the  decision-making  criteria  for  a 
particular  breakthrough  not  only  the  objectives  to  be 
achieved  but  also  the  difficulties  likely  to  be  encoun- 
tered and  ways  of  preventing  or  minimizing  such 
difficulties.  This  requires  consideration  of  possible 
undesirable  side  effects,  limitations  imposed  by  re- 
sources reasonably  available,  and  people-generated 
stalls,  objections,  and  roadblocks. 

— Learning  how  to  cope  with  breakthrough-type 
tasks  by  starting  with  a  relatively  small  task  on  a  sort 
of  trial-run  basis.  Success  in  accomplishment  of  this 
task  should  be  followed  by  progressively  larger  tasks, 
to  increase  the  skill  in  handling  breakthrough-type 
work.  This  gradual  approach  also  helps  to  build  con- 
fidence— a  vital  factor  for  attempting  any  type  of 
breakthrough  work  [3] . 

— Screening  the  alternatives  by  systematically  check- 
ing them  against  the  objectives,  undesirable  side  ef- 
fects, resources  and  people-generated  stalls,  objections, 
and  roadblocks.  In  addition,  objectives  should  be 
categorized  as  either  "musts"  or  "wants"  [4] . 

— Recognizing  the  importance  of  motivation  in  get- 
ting people  to  work  effectively  on  the  breakthrough 
project.  Motivation  can  be  developed  and  improved, 
and  in  this  way  much  of  the  dormant  creative  potential 
and  other  problem-solving  skills  of  people  can  be  re- 
leased and  effectively  focused  on  the  main  problems 
and  subproblems  required  to  achieve  the  desired 
breakthrough  [2] . 
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6.  Possibilities  for  Application  of 
Breakthrough  Techniques  to 
Metrology  Problems 

Several  areas  which  suggest  themselves  as  good  pos- 
sibilities for  application  of  the  breakthrough  tech- 
niques discussed  in  this  paper  are  as  follows: 

•  The  calibration  recall  problem. 

•  The  problem  of  traceability  of  calibration  to  NBS. 

•  Improved  utilization  of  measuring  equipment  and 
standards. 

•  How  to  obtain  a  compatible  system  of  measure- 
ments and  tests  throughout  the  United  States. 


7.  References 

[1]  Juran,  J.  M.,  Managerial  Breakthrough  (McGraw-Hill,  New 

York,  N.Y.,  1964) . 
[2]  Williams,  J.  Clifton,  Dynamics  of  Motivational  Management 

(Success  Motivation  Institute,  Waco,  Tex.,  1969) . 
[3]  Bobbe,  Richard  A.,  and  Schafer,  Robert  H.,  Mastering 

change:  Breakthrough  projects  and  beyond,  Management 

Bull.  No.  120  (Am.  Management  Assn.,  New  York,  N.Y., 

1968). 

[4]  Kepner,  Charles  H..  and  Tregoe,  Benjamin  B.,  The  Rational 
Manager  (McGraw-Hill,  New  York,  N.Y.,  1965). 


NCSL  70 


THE  NEW  STATE  STANDARDS  PROGRAM 
Robert  Mills 

Office  of  Weights  and  Measures,  National  Bureau  of  Standards,  Washington,  D.C.  20234 

In  1965,  Congress  appropriated  funds  for  new  state  standards  and  instruments  in  mass,  length,  and 
volume.  The  laboratories  receiving  this  package  are,  in  most  cases,  under  the  administration  of  state 
offices  of  weights  and  measures.  These  labs  not  only  check  the  accuracy  of  field  standards  used  by 
weights  and  measures  inspectors,  but  also  serve  as  local  measurement  centers  to  perform  industrial 
tests.  The  equipment  and  training  given  to  laboratory  personnel  are  described  in  this  paper,  as  well 
as  some  of  the  administrative  details  and  the  utilization  we  foresee  for  the  state  laboratories. 
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1.  Introduction 

In  1965  the  United  States  Congress  approved  a  two 
million  dollar  program  to  provide  states  with  new 
physical  standards  and  measuring  instruments  in 
mass,  length,  and  volume.  The  National  Bureau  of 
Standards  was  charged  with  the  responsibility  of 
assisting  the  states  in  the  development  of  state  lab- 
oratories, design  and  procurement  of  standards  and 
precision  balances,  and  final  installation.  Today  40 
states  are  participating  in  the  State  Standards 
Program. 

These  newly  equipped  metrology  laboratories  en- 
able the  states  to  assume  their  important  role  in  our 
national  measurement  system  during  a  time  when  sci- 
ence and  technology  are  advancing  at  an  accelerating 
rate.  It  is  important  that  the  membership  of  the 
National  Conference  of  Standards  Laboratories  be 
informed  about  the  New  State  Standards  Program. 
Thus,  it  is  my  pleasure  to  discuss  with  you  the  pro- 
gram's origin,  some  of  the  administrative  details,  and 
the  utilization  we  foresee  for  the  state  laboratories. 

Men  have  endeavored  to  establish  effective  stand- 
ards of  measurements  for  many  centuries.  Edward  II 
proclaimed  in  1324  that  the  inch  was  equal  to  the 
length  of  three  barley  corns,  taken  from  the  center  of 
an  ear,  placed  end  to  end.  In  the  16th  century,  the 
rod  was  the  length  of  the  left  feet  of  16  men  lined  up 
as  they  left  church  on  Sunday  morning.  Perhaps 
these  standards  sufficed  in  their  time,  but  it  is  clear 
that  they  would  not  fulfill  today's  need  for  standardi- 
zation. 

In  our  country,  there  has  long  been  a  parallel  be- 
tween the  existence  of  accurate  standards  and  the  es- 
tablishment of  effective  weights  and  measures  activi- 
ties. The  Constitution  of  the  United  States  empowered 
Congress  in  Article  1,  Section  8  to  "fix  the  standards 
of  weights  and  measures."  In  1821,  John  Quincy 
Adams  recommended  that  Congress  act  to  bring  about 


uniformity  in  weights  and  measures.  The  Senate 
passed  a  resolution  in  1830  directing  the  Department 
of  the  Treasury  to  compare  the  standards  of  weights 
and  measures  in  use  at  the  principal  customhouses. 
Large  discrepancies  were  found  to  exist.  As  a  result, 
Congress  directed  the  Treasury  to  fabricate  standards 
of  weights  and  measures  for  the  customhouses  and 
States,  thereby  authorizing  the  distribution  of  the  first 
State  standards.  This  resolution  also  established  the 
Office  of  Weights  and  Measures.  Two  additional  dis- 
tributions of  standards  to  the  States  were  authorized 
by  Congress  in  1866  and  1881. 

By  the  beginning  of  the  20th  century,  the  Industrial 
Revolution  had  produced  a  growing  awareness  of  the 
need  for  uniform,  nationwide  measurement  techniques 
and  measurement  standards  in  the  United  States.  The 
Office  of  Weights  and  Measures  was  transferred  to 
the  National  Bureau  of  Standards  in  1901,  the  year 
the  Bureau  was  established. 

Traditionally,  regulatory  authority  in  weights  and 
measures  rests  with  the  States  rather  than  with  the 
Federal  Government.  There  are  50  state  weights  and 
measures  jurisdictions,  and  in  some  cases  local  author- 
ities within  the  States,  separately  exercising  their  au- 
thority to  check  supermarket  scales,  gasoline  pumps, 
packages  in  the  marketplace,  taximeters,  fabric  and 
cordage  measuring  devices,  farm  milk  tanks,  truck 
scales,  fuel  oil  meters — virtually  every  weighing  and 
measuring  device  used  commercially.  Yet,  despite 
this  diversification  of  responsibility,  measurement 
accuracy  and  uniformity  must  be  maintained  to  pro- 
tect both  the  buyer  and  seller,  and  to  provide  for 
orderly  commerce  on  a  national  basis.  Uniformity 
in  specifications,  tolerances,  and  test  procedures  is 
achieved  in  most  states  by  accepting  the  recommenda- 
tions [1]  1  of  the  National  Conference  on  Weights 


1  Figures  in  brackets  refer  to  the  literature  references  at  the  end 
of  this  paper. 
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and  Meaures,  another  conference  sponsored  by  NBS. 
Accurate  field  test  standards  must  also  be  used  by 
weights  and  measures  inspectors.  This  requirement 
implies  the  need  for  accurate  laboratory  standards. 
By  1965,  the  laboratory  standards  of  many  States  were 
no  longer  adequate. 

2.  The  Program 

Congress  responded  to  the  need  for  State  reference 
standards  by  establishing  the  State  Standards  Pro- 
gram. The  National  Bureau  of  Standards  was  given 
the  responsibility  of  acquiring  the  new  standards  and 
instruments,  training  State  laboratory  personnel,  and 
assisting  the  States  to  provide  adequate  laboratory 
facilities.  Primary  responsibility  for  the  project  lies 
with  the  NBS  Office  of  Weights  and  Measures,  which 
offers  assistance  and  technical  support  to  local  and 
State  weights  and  measures  agencies.  The  NMS  Me- 
trology Division  performs  the  calibrations  of  the  new 
standards  and  provides  technical  counsel  when 
needed. 

States  initiate  the  process  of  qualifying  for  receipt 
of  the  State  Standards  "package"  by  demonstrating 
the  need,  and  by  making  a  commitment  to  provide 
suitable  laboratory  space  and  personnel.  NBS  then 
discusses  details  of  requirements  for  laboratory  facili- 
ties with  State  architects  or  engineers,  and  provides 
additional  technical  assistance  as  construction  pro- 
ceeds. The  construction  may  be  modifications  to 
existing  facilities  or  the  erection  of  a  completely  new 
building.  Construction  costs  to  the  states  have  ranged 
from  a  few  thousand  dollars  up  to  about  .$135,000.  A 
minimum  of  1,000  square  feet  is  needed  for  a  suitable 
laboratory  facility. 

The  new  standards  have  been  furnished  at  the  rate 
of  approximately  10  states  per  year.  To  date,  for- 
mal presentation  ceremonies,  usually  attended  by  the 
NBS  Director  and  by  the  Governor,  have  been  held 
in  23  states.  Table  1  lists  the  states  in  each  group 
and  the  dates  of  the  formal  presentation  ceremonies 
which  have  been  held.  The  grouping  of  the  states 
was  determined  by  their  needs,  and  the  date  on  which 
they  provided  the  required  laboratory  facilities  and 
personnel. 

The  Office  of  Weights  and  Measures  of  NBS  pro- 
vides training  for  laboratory  personnel  at  three  levels: 
basic,  intermediate,  and  advanced.  The  two  weeks 
of  basic  training  are  completed  before  the  official  pres- 
entation of  standards.  The  first  week  is  conducted 
at  the  NBS  Office  of  Weights  and  Measures  labora- 
tory, or,  to  accommodate  distant  States,  at  established 
State  laboratories,  usually  just  prior  to  the  presenta- 
tion ceremony  and  concurrently  with  the  final  installa- 
tion of  the  new  laboratory  equipment.  Metrologists 
who  are  replacements  for  previously  trained  personnel 
receive  their  basic  training  at  NBS. 

Intermediate  seminars  are  intended  for  State  metrol- 
ogists who  have  completed  basic  training.  These 
seminars  are  usually  40  hours  long  and  can  be  held 
either  at  NBS  or  in  regional  seminars.  They  begin 
by  reviewing  basic  subjects  before  continuing  to  more 
advanced  material.    The  advanced  seminars  are  for 


Table  1.    Progress  in  presentation  of  new  standards 
to  the  50  States 


Presentation 

Presentation 

Group  I 

Date 

Group  II 

Date 

- 

Crfciliiornia. 

1 1  / 99 /A7 
11/  ZZ/  0  1 

A 

Arkansas 

a  1 1  o  /  /en 

4/ io/oy 

Connecticut 

3/  A/fift 
0/    <*/  Do 

Florida 

Delswfire 

1  /  zAjf  Uu 

Georgia 

1  A  /  Ol  /  AO 

iU' zi/ oy 

Illinois 

f\  /Q/fn 
o  /  y/  o  i 

Hawaii 

Q/   9 /AO 
o/  Z/ Oo 

Kentucky 

7/19/68 

Maine 

5/21/69 

New  Mexico 

12/  8/67 

Ohio 

6/  8/67 

North  Carolina 

2/25/69 

Oregon 

10/27/67 

Pennsylvania 

3  /3/69 

Tennessee 

o/   z/  uo 

^^est  Virginia 

c;/ iQ/7fl 

Utah 

in/ v>/fi7 

\X/  ic/>nncin 

3/  7/6.Q 
0/    i  /  \jy 

Presentation 

Presentation 

Group  III 

Date 

Group  IV 

Date 

Alabama 

A 

Alaska 

Idaho 

£  /   A  /7fi 

Colorado 

Indiana 

Massachusetts 

Maryland 

A  /  A.ll(\ 
41/    Hel  /U 

Michigan 

New  Jersey 

Minnesota 

North  Dakota 

Nevada 

Oklahoma 

New  York 

Texas 

6/  4/70 

South  Carolina 

Vermont 

South  Dakota 

Wyoming 

4/13/70 

Virginia 

Remaining  States  and  Territories 


Arizona 

Iowa 

Kansas 

Louisiana 

Mississippi 

Montana 

Nebraska 

New  Hampshire 

Rhode  Island 

Washington 

District  of  Columbia 
Puerto  Rico 
Virgin  Islands 


more  experienced  metrologists.  They  are  held  at  NBS 
and  provide  a  broad  exposure  to  the  Bureau's  pro- 
fessional staff.  Some  thought  has  been  given  to  sched- 
uling advanced  seminars  the  week  just  before  or  just 
after  future  NCSL  dates  so  that  State  metrologists 
can  attend  this  Conference. 

Our  intermediate  and  advanced  training  programs 
are  continuous  in  that  metrologists  are  encouraged  to 
attend  a  seminar  once  every  year.  Participants  are 
carefully  selected  for  each  seminar  so  that  experience 
and  level  of  achievement  are  compatible.  Seminars 
at  all  three  levels  have  had,  on  the  average,  four  or 
five  participants.  We  expect  to  continue  our  present 
schedule  of  from  10  to  12  seminars  a  year. 

3.  State  Standards 

Table  2  lists  the  standards  and  instruments  in  the 
State  Standards  "package."  Note  that  both  metric 
and  avoirdupois,  or  U.S.  customary  standards,  are 
included.    The  mass  standards  in  item  (1)  and  (2) 
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comprise  a  5,  3,  2,  1  series  (e.g.,  5  g,  3  g,  2  g,  1  g, 
0.5  g,  etc)  with  duplicate  1  kg,  1  mg,  1  lb,  and  1  yJb 
weights.  The  small  flat  weights  (i.e.,  smaller  than 
1  g  and  0.002  lb )  are  made  of  Nichrome  or  alumi- 
num. Except  for  the  500  lb  standards,  which  are  303 
stainless  steel,  the  larger  weights  are  made  of  a  stain- 
less steel,  developed  especially  for  precision  mass 
standards,  having  a  density  of  8.0  g/cm3.  The  syste- 
matic uncertainties  in  the  reports  of  calibration  range 
from  30  mg  for  the  30  kg  weight  to  2.9  [xg  for  the 


1  mg  weight.  The  five  precision  balances  are  the  best 
present  technology  offers.  The  precision  given  in  ta- 
ble 2  are  manufacturers'  tolerances  and  are  included 
to  indicate  the  capabilities  of  these  instruments.  Ex- 
cept for  the  5,000  lb  equal-arm  Russell  balance,  all  of 
these  instruments  are  single-pan,  substitution  balances. 
Prior  to  the  beginning  of  this  program,  the  standards 
and  balances  were  developed  through  the  cooperative 
efforts  of  NBS  and  the  manufacturers.  Reference 
[2]  discusses  the  development  of  the  30  kg  balance. 


Figure  3.  The  30  kg  and  Russell  balance. 


Table  2.    State  Standards 


1.  METRIC  MASS  STANDARDS— 30  kg  to  1  mg,  fig.  1 

2.  AVOIRDUPOIS  MASS  STANDARDS— 50  lb  to  1  n\b 

3.  500  LB  MASS  STANDARDS  (2) 

4.  PRECISION  BALANCES,  figs.  2  and  3: 

160  g  capacity — 0.02  mg  precision 
1  kg  capacity — 0.2  mg  precision 
3  kg  capacity — 1  mg  precision 
30  kg  capacity — 2  mg  precision 
5000  lb  capacity — 0.01  mg  precision 

5.  LENGTH  BENCH— 5  meter/ 16  feet 

6.  TENSION  WEIGHTS— 20  lb 

7.  LABORATORY  MICROSCOPE— 0.300  inch  X  0.002  inch 

8.  PRECISION  STEEL  TAPE— 7  meter/25  feet 

9.  STEEL  TAPE— 30  meter/ 100  feet 

10.  PRECISION  STEEL  RULE— 18  inches  X  0.01  inch 

11.  METRIC  PIPET-BURET  ASSEMBLY— 5  liters  to  10  mil- 

liliters 

12.  U.S.  CUSTOMARY  PIPET-BURET  ASSEMBLY— 1  gallon 

to  120  minims 

13.  FIVE  GALLON  STANDARD— slicker  plate  type 


Items  (8)  and  (10)  are  the  state  primary  length 
standards.  The  report  of  calibration  for  the  25  ft/7m 
tape  gives  an  uncertainty  of  0.002  in.  and  0.05  mm  for 
any  interval.  Uncertainty  of  the  18  in.  scale  is  re- 
ported as  0.001  in.  For  the  volumetric  standards,  the 
reported  uncertainty  is  0.001  gal  for  the  5-gal  slicker 
plate  measure.  0.1  minim  for  the  120-minim  buret,  and 
0.005  ml  for  the  l()-ml  buret. 


4.  Utilization  of  Laboratories 

The  primary  role  of  a  State  measurement  center  is 
to  serve  as  a  reference  laboratory  for  weights  and 
measures  field  standards.  In  almost  every  case,  the 
State  standards  equipment  is  received  by  the  State 
weights  and  measures  department.  State  laboratories 
also  check  the  standards  used  by  other  Stale  agencies 
and  officials,  such  as  the  state  chemist;  veterinarian; 
dairy,  feed,  and  fertilizer  inspector;  public  health  in- 
spector; highway  department;  and  petroleum  labora- 
tory. 

From  the  outset  there  was  no  intention  to  restrict 
activities  of  the  State  Standards  Laboratories  to 
weights  and  measures  activities,  or  to  other  state  reg- 
ulatory agencies.  Rather,  the  policy  has  been  to  en- 
courage states  to  serve  within  their  capability  the 
measurement  needs  of  industry  and  commerce  as  well 
as  educational  and  research  institutions. 

There  was  evidence  of  an  industrial  demand  at  State 
metrology  laboratories  even  before  the  initiation  of  the 
State  Standards  Program.  Massachusetts,  for  one,  has 
had  an  active  program  of  providing  calibrations  for 
their  industries.  They  are  just  now  beginning  to  re- 
ceive their  new  standards,  and  anticipate  an  increase 
in  both  the  quality  and  quantity  of  the  measurement 
services  provided  to  industry.  The  demand  for  in- 
dustrial calibrations  at  the  State  laboratories  has  been 
made  apparent  by  the  activities  of  some  of  the  labs 
which  have  received  their  new  standards.  Arkansas  is 
a  good  case  in  point.  Table  3  indicates  the  scope  of 
the  industrial  tests  conducted  there  in  addition  to  tests 
of  their  own  field  standards  and  tests  for  other  State 
agencies.    Admittedly,  the  development  of  the  Arkan- 
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sas  laboratory  is  exceptional.   Few  industrial  tests  were 
I  performed  before  the  new  Arkansas  standards  were 
|  presented  in  April  of  1969;  the  table  covers  the  period 
i  up  to  March  1970.    Of  course,  Arkansas  is  not  one  of 
our  most  industrialized  States.    Several  other  States 
having  a  more  industrialized  economy,  such  as  Cali- 
fornia, Illinois,  or  Pennsylvania,  perform  even  more 
industrial  tests. 

Most  States  charge  fees  for  the  industrial  tests  they 
perform.  Thus,  costs  are  paid  by  industries  which 
have  the  need  for  tests  with  standards  traceable  to 
NBS.  Of  course,  the  States  pay  the  costs  for  testing 
their  own  field  standards. 


Table  3.  Arkansas  weights  and  measures  laboratory 
industrial  customers 


1. 

Electronic 

Weights  and  weight  kits  from  2  kg  to  1  g 

Companies 

(Class  S)  and  50  lb  to  1/32  oz  (Class  F) . 

2. 

Petroleum 

Metal  volumetric  test  measures  from  1  gal 

Equipment 

to  500  gal.    All  tests  were  certified  to  1 

Companies 

part  in  2000. 

3. 

Pipe  Line 

Metal  test  provers;  two  600  gal  and  one 

Companies 

1,015  gal.  Accuracy  1  part  in  2000. 

4. 

Bottling 

Testing  of  graduated  neck  flasks  used  to 

Companies 

spot  check  fill  machines. 

5. 

Scale 

Weights  and  weight  kits  from  1/32  oz  to 

Companies 

1.000  lb  and  1  g  to  25  kg.  Class  S,  F,  and 

T  tolerances  were  required 

6. 

Pharmaceutical  Weights  and  weight  kits  from  1  mg  to  20 

Laboratories 

kg.  Class  S  tolerances. 

7. 

Electrical 

Various  weights,  Class  T  tolerance. 

Companies 

8. 

Oil  Refining 

Calibration  of  tank  trucks:  Accuracy  1  part 

Companies 

in  2000. 

9. 

Manufacturing 

Weight  kits  from  1  mg  to  5  kg:  Class  S 

Companies 

tolerance  was  requested. 

10. 

Milling 

Weights  from  4  oz  to  5  lb;  Class  F  toler- 

Companies 

ance  were  requested. 

11. 

Transport 

Calibration  of  tank  trucks:  Accuracy  1  part 

Companies 

in  2000. 

Note:  The  Arkansas  Laboratory  has  seven  large-capacity  provers 
having  nominal  capacities  between  29  and  1,000  gallons. 


In  the  training  program,  State  metrologists  are  en- 
couraged to  consult  with  NBS  about  tests  that  are  other 
than  routine.  Our  office  has  a  policy  of  being  as  help- 
ful as  possible  in  such  situations.  Jobs  which  are  be- 
yond the  capabilities  of  particular  State  laboratories 
are  referred  to  NBS.  Clearly  it  is  important  to  know 
the  capabilities  of  each  State  laboratory.  This  sur- 
veillance is  accomplished,  at  present,  through  the  Lab- 
oratory Auditing  Program  (LAP),  which  provides  for 
tests  to  monitor  the  performance  of  the  instruments, 
personnel,  and  environment.  Table  4  indicates  the  na- 
ture of  LAP  exercises  by  listing  the  titles  of  the  first 
ten  problems.  Problems  4  and  9  involve  the  exchange 
of  test  kits  between  our  laboratory  and  the  States. 
Data  from  all  LAP  exercises  are  sent  to  us  for  review 
and  comments.    Certificates  are  awarded  annually  to 


States  which  have  met  minimum  requirements  of  par- 
ticipation. 

Table  4.  Laboratory  auditing  program  problems 


1  Sensitivity  test 

2  Tolerance  test  (direct-reading) 

3  Precision  test  of  1-kg  balance 

4  Tolerance  test  and  weight  calibration 

5  100-g  balance  precision 

6  3-kg  balance  precision 

7  30-kg  palance  precision 

8  Calibration  of  length  bench 

9  Testing  of  a  steel  tape 

10  Precision  of  Russell  balance 


5.  The  Future 

The  surveillance  effort  will  have  to  be  increased  as 
the  measurements  performed  by  the  State  laboratories 
become  more  sophisticated.  States  should  record  a 
continuous  flow  of  surveillance  data  so  that  control 
charts  can  be  maintained.  The  more  advanced  State 
metrologists  are  presently  doing  this.  They  receive 
an  analysis  of  their  control  chart  data  from  the  Office 
of  Weights  and  Measures.  In  this  way,  the  capability 
of  their  laboratory  can  be  determined  realistically.  Our 
intention  is  to  make  this  a  more  universal  practice  in 
the  State  laboratories. 

Insufficient  surveillance  data  will  result  in  a  large 
uncertainty  as  to  a  State  laboratory's  performance  ca- 
pability. The  proper  surveillance  effort  will  be  deter- 
mined by  the  calibration  demands  on  that  laboratory. 
Time  required  to  accumulate  surveillance  data  will 
probably  range  from  one  to  ten  man-hours  per  week 
(not  necessarily  time  spent  by  the  metrologists;  data 
can  be  taken  and  compiled  by  an  aide) . 

There  has  been  an  increasing  number  of  inquiries 
recently  concerning  the  possible  extension  of  State 
laboratories  into  fields  of  measurement  other  than  mass, 
length,  and  volume.  The  greatest  interest  is  in  tem- 
perature and  gage-block  measurements.  The  Massa- 
chusetts laboratory  has  been  in  the  business  of  testing 
thermometers  for  some  time.  California  is  testing 
electric  meters  in  addition  to  other  measurements.  We 
at  the  National  Bureau  of  Standards  are  preparing  to 
advise  the  States,  upon  request,  concerning  test  equip- 
ment, test  procedures,  and  test  uncertainties  in  fields 
where  the  need  can  be  demonstrated. 

Getting  back  to  the  State  Standards  Program,  the 
future  offers  a  triple  challenge  to  the  states,  to  the  Na- 
tional Bureau  of  Standards,  and  to  potential  industrial 
users  if  full  utiliztion  of  these  State  measurement  cen- 
ters is  to  be  realized.  The  States  must,  of  course  pro- 
vide adequate  financial  and  administrative  support  for 
their  laboratories.  They  must  staff  their  laboratories 
with  high-caliber  personnel.  The  National  Bureau  of 
Standards  must  maintain  a  vigorous  support  program 
in  training  and  consulting  services.  Potential  indus- 
trial customers  must  be  informed  about  their  State 
laboratory  and  make  available  information  on  their 
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future  measurement  requirements.  Industrial  interest 
and  support  can  be  helpful  in  encouraging  the  states 
to  properly  fulfill  many  of  their  measurement  require- 
ments. The  NBS  Office  of  Weights  and  Measures  in- 
vites inquiries  from  any  interested  parties  concerning 
the  current  status  of  the  laboratory  program  in  their 
particular  state.  Comments  and  suggestions  are,  of 
course,  welcome. 
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In  the  present  economic  environment,  the  applica- 
tion of  new  and  improved  methods  of  visibility  and 
control  in  the  standards  and  calibration  laboratory  be- 
comes a  high-priority  goal.  The  methods  of  visibility 
and  control  we  are  going  to  explore  have  evolved  over 
a  period  of  years  and  are  presently  operational  in  the 
Standards  and  Calibration  Laboratories  at  the  General 
Dynamics  Convair  Division.  They  were  developed 
during  the  best  possible  environment  for  designing  a 
control  system;  namely,  a  continuing  decrease  in  the 
departmental  budget,  an  increase  in  responsibility,  and 
a  demand  for  continued  product  quality.  Such  condi- 
tions certainly  cause  one  to  look  around  for  new  con- 
trols to  maintain  quality  and  optimize  cost! 

Understandably,  the  methods  developed  are  tailored 
to  Convair  operations.  Convair  calibration  labora- 
tories may  or  may  not  be  similar  to  those  in  other 
companies,  but  one  thing  is  certain  .  .  .  many  of  the 
problems  and  the  customer  requirements  will  be  very 
familiar  to  everyone. 

Before  we  discuss  these  methods,  it  will  be  necessary 
to  acquaint  you  with  Convair's  basic  philosophy  of  op- 
eration and  organization  of  calibration  activities.  The 
Standards  and  Calibration  Laboratories  are  a  depart- 
ment of  Reliability  Control.  They  are  further  divided 
into  three  sections;  Electronic  Metrology,  Mechanical 
Metrology,  and  Test  Equipment  Control  Engineering. 
The  Electronic  and  Mechanical  Metrology  sections  in- 
clude the  calibration  laboratories.  These  laboratories 
are  further  segregated  into  10  major  laboratory  areas 
and  21  subgroups  identified  by  the  category  of  equip- 
ment they  service.  Each  laboratory  is  operated  under 
the  technical  direction  of  a  graduate  engineer,  with 
highly  skilled  technicians  doing  the  actual  calibration 
and  maintenance.  The  Engineer-in-Charge  of  each 
laboratory  approves  all  work  done.  He  certifies  the 
work  by  means  of  a  Reliability  Control  Department 
metrology  stamp  assigned  to  him.    There  are  no  reg- 


ular inspection  department  or  other  quality  control 
personnel  assigned  to  the  standards  and  calibration 
laboratories. 

The  laboratories  under  the  Electronic  and  Mechani- 
cal Metrology  sections  service  approximately  40,000 
items  of  test  and  measuring  squipment,  of  which  ap- 
proximately 30,000  items  are  in  continuous  use  in  Con- 
vair operations. 

The  Test  Equipment  Control  Engineering  section  is 
the  group  most  pertinent  to  this  discussion,  since  it 
plays  a  dynamic  role  in  the  control  system.  Conse- 
quently, though  each  section  is  involved  in  control  to 
some  extent,  we  will  examine  the  responsibilities  and 
operations  of  the  Test  Equipment  Control  Engineering 
section  in  detail.  The  primary  responsibilities  of  the 
Test  Equipment  Control  Engineering  section  are  to: 

1.  Develop  and  provide  policy  procedures  for  the 
entire  Standards  and  Calibration  Laboratories  Depart- 
ment. 

2.  Respond  to  Convair  Division  requirements  for 
general-purpose  measuring  and  test  equipment,  whether 
for  new  applications  or  replacement  purposes.  This 
response  includes  "cradle  to  the  grave"  activities,  such 
as  acquisition,  control,  storage,  and  eventual  final  dis- 
position of  the  equipment. 

3.  Provide  a  central  record  control  system  for  all 
work  entering  or  leaving  the  Department. 

4.  Maintain  all  calibration  history  and  traceability 
records  for  each  item  of  equipment  and  provide  cali- 
bration recall  schedules. 

5.  Establish  all  calibration  intervals  and  initiate  or 
approve  any  status  change  in  measuring  or  test  equip- 
ment. 

6.  Maintain  general-purpose  measuring  and  test 
equipment  issuance  stockrooms  at  practical  and  appro- 
priate locations  throughout  the  Division. 

7.  Provide  engineering  technical  support  for  the  so- 
lution of  Division  measurement  problems. 
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To  fulfill  these  assignments,  the  Test  Equipment  Con- 
trol Engineering  section  is  composed  of: 

1.  A  part-time  staff  technical  and  management  policy 
writer. 

2.  A  records  and  data  group. 

3.  A  test  equipment  stockroom  operations  group. 

4.  A  test  equipment  applications  engineering  group. 
All  applications  engineers  are  graduate  engineers  with 
laboratory  experience. 

Now.  let's  discuss  visibility  and  control  problems 
and  solutions.  The  question  may  be  raised  as  to  why 
the  "cradle  to  the  grave"  responsibility  for  general- 
purpose  measuring  and  test  equipment  is  vested  in  a 
calibration  laboratory  operation.  Why?  Because 
this  concept  permits  overall  visibility  and  control  lead- 
ing directly  to  continuous  cost  optimization  opportu- 
nities for  the  company.  Let's  investigate  the  truth  of 
this  statement. 

If  a  company  allows  any  individual  user  or  designer 
to  specify  his  test  equipment  preferences  by  manufac- 
turer and  model,  a  high-cost  chain  reaction  is  set  in 
motion.  Start  with  the  unit  itself.  It  may  not  meet 
the  task  requirements  or  even  its  own  published  specifi- 
cations. Get  your  money  back?  Hardly.  The  cost  of 
the  unit  is  probably  a  fraction  of  the  cost  of  main- 
taining the  production  schedules  involved  in  our  busi- 
nesses. Next,  consider  spares.  Spares  must  be  pur- 
chased for  all  the  different  models  that  are  specified 
and  subsequently  purchased.  Then  calibration  proce- 
dures must  be  obtained  for  all  different  models.  Cali- 
bration equipment  must  be  obtained  for  all  different 
models.  By  now,  our  chain  reaction  is  well  under  way. 
Calibrating  and  operating  technicians  must  be  trained 
on  the  different  models.  There  is  also  the  possibility 
that  once  the  unit  has  served  its  original  purpose  it  may 
be  useless  for  other  applications.  These  considerations 
are  seldom  considered  by  the  individual  designer  or 
test  equipment  user,  simply  because  he  lacks  an  overall 
view.  . 

What  is  necessary  for  a  group  to  provide  a  rational, 
standardized  test  equipment  program  for  a  large  com- 
pany? It  must  have  complete  responsibility  and  con- 
trol over  the  entire  program.  It  must  have  expert  tech- 
nical knowledge  of  all  measuring  and  test  equipment 
used  in  the  company  and  of  the  costs  of  maintaining 
such  equipment.  It  must  have  intimate  knowledge  of 
vendor  reputations  in  the  various  fields,  gained  from 
actual  instrument  comparisons.  It  must  be  able  to 
communicate  on  the  technical  level  of  the  engineer. 

Where  can  you  find  a  group  with  these  capabilities? 
If  your  calibration  laboratories  are  manned  with  grad- 
uate engineers,  each  one  a  technical  expert  on  a  par- 
ticular category  of  equipment,  which  he  also  maintains 
for  the  company  .  .  .  then  you  certainly  have  the  base 
for  such  a  group. 

As  an  example  of  what  can  be  accomplished,  the 
test  equipment  applications  group  developed  comput- 
er lab-run  listings  of  all  test  equipment  in  the  Convair 
Division.  Results  showed  7.326  different  models  in 
the  40,000  items  of  equipment.  By  working  closely 
with  users  to  determine  application  requirements,  and 
with  calibration  laboratory  engineers  regarding  specifi- 
cations and  choice  of  equipment,  it  was  determined 


that  only  2,179  different  models  were  actually  required.  : 
And  this  was  only  the  first  look!  Some  of  the  5,147 
unwanted  models  were  replaced  immediately  by  pre- 
ferred models  that  were  in  storage  status.  The  remain- 
ing unwanted  models  were  marked  for  surplus  on  a 
systematic  basis.  This  was  accomplished  by  establish- 
ing a  zero  retention  level  for  the  unwanted  units  on 
the  storage  control  lists.  Consequently,  if  a  user  does 
not  have  an  immediate  need  for  a  unit,  it  is  declared 
surplus  rather  than  being  stored  for  backup  purposes. 

On  the  subject  of  test  equipment  disposition,  how 
many  of  us  have  pondered  over  the  question  of  wheth- 
er to  dispose,  or  not  to  dispose,  of  test  equipment? 
More  to  the  point  is  the  pertinent  question:  Who  makes 
the  decision?  Certainly  the  decision  to  use,  store,  or 
dispose  of  a  unit  of  test  equipment  will  either  save,  or 
cost,  the  company  money.  There  is  no  middle  ground. 
It  will  be  one  or  the  other.  Without  a  capable  control 
group,  the  decision  is  usually  made  by  those  least 
qualified  to  make  it,  usually  a  test  equipment  stock- 
room operation.  Furthermore,  their  decision  is  usually 
to  store  the  unit,  since  this  is  a  pretty  safe  decision. 
In  fact,  the  buck  can  get  passed  pretty  far  up  the  man- 
agement ladder  before  someone  will  commit  himself 
on  this  apparently  simple  decision.  Why  does  this 
hold  true?  Because  no  company-wide  visibility  on  the 
matter  is  available  to  the  decision  maker;  therefore,  no 
decision  rules  have  been  formulated. 

About  three  years  ago  the  Test  Equipment  Control 
Engineering  section  was  confronted  with  this  problem 
by  one  of  our  customers.  The  customer  pointed  out 
to  management  various  contractual  clauses  which  re- 
quired customer-owned  equipment  to  be  returned  with- 
in a  certain  period  of  time  after  it  had  been  declared 
excess.  Moreover,  he  claimed  we  were  keeping  his 
equipment  in  storage  for  years  as  potential  backup 
equipment.  Unfortunately,  the  allegations  proved  true. 
A  fast  check  showed  that  not  only  was  the  storage  in- 
ventory huge,  it  was  snowballing  rapidly.  The  inven- 
tory included  both  customer  and  company  equipments 
which  a  cursory  investigation  showed  were  worthless 
for  future  requirements.  Economically,  the  inventory 
was  almost  a  disaster.  The  lesson  was  driven  deeper 
when  other  equipment  was  located  in  storage  that  was 
superior  to  equipment  in  use. 

Today,  the  decision  to  use,  store,  or  declare  equip- 
ment surplus  is  made  within  one  to  two  days  after  the 
user  decides  the  unit  is  excess  to  his  needs.  The  de- 
cision is  made  as  a  matter  of  routine,  with  complete 
confidence  that  it  is  correct.  An  engineer  from  the 
test  equipment  applications  group  uses  management- 
designed  decision  rules,  along  with  engineering  judg- 
ment regarding  the  condition  of  the  instrument,  to  ar- 
rive at  his  decision  within  a  few  minutes  after  the  ques- 
tion arises. 

Visibility  makes  this  possible:  visibility  of  all  test 
equipment  in  the  company  and  where  it  is,  visibility 
of  present  and  known  future  needs  of  all  users  in  the 
company.  Visibility  of  preferred  test  equipment.  Visi- 
bility of  the  desired  number  of  like  units  for  backup 
capability,  the  current  headcount  of  like  units  in  stor- 
age, and  the  precise  condition  of  each  one. 

Of  course,  we  are  talking  about  a  computerized  test 
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equipment  control  system  with  inputs  of  all  parameters 
of  test  equipment  cost,  control,  and  life-history  on 
every  interface  of  the  equipment  with  operations.  Out- 

I  puts  are  obtained  in  the  form  of  tab  runs  and  listings, 
in  the  formats  required  for  intelligent  decision-rule 

■  formulation  or  equipment  control.  These  outputs  are 
then  placed  in  the  hands  of  all  those  requiring  the  in- 
formation for  decision  making  or  equipment  control. 
This  point  may  seem  trite,  but  all  too  often  informa- 
tion is  available  within  a  company  but  its  existence 
is  unknown  to  the  poor  soul  who  really  needs  it. 

Some  of  the  management  tools  provided  by  the 
computerized  test  equipment  control  system  are: 

1.  Purchase  cost  of  the  unit. 

2.  Maintenance  cost  for  the  unit. 

3.  Chronological  history  of  the  unit,  including  num- 
bers and  dates  of  calibration,  numbers  and  dates  of 
failures,  types  of  failures,  and  whether  or  not  the 
failure  was  detected  by  the  user. 

4.  User  and  location  of  equipment. 

5.  Equipment  stored  and  whether  operational  or 
needing  repair.  Also  the  maximum  number  of  each 
type  to  be  held  in  storage. 

6.  Calibration  and  maintenance  time  on  all  equip- 
ment serviced  by  the  individual  technician. 

7.  Preferred  test  equipment  for  standardization. 

8.  Backlogs  of  individual  laboratories. 

9.  Future  workload,  both  total  and  individual  labo- 
ratories. 

Inputs  regarding  user  requirements,  immediate  and 
future,  are  also  provided  to  the  section  by  means  of 
test  equipment  requests. 

Management  use  of  these  tools  in  their  various  pos- 
sible combinations  and  forms  will: 

1.  Provide  the  Standards  and  Calibration  Labora- 
tory management,  higher  management,  and  the  user 
with  actual  cost  information,  including  test  equipment 
calibration  and  maintenance  costs  for  each  user  in  the 
company. 

2.  Provide  calibration  recall  listings  for  all  users  and 
the  various  calibration  laboratories. 

3.  Determine  the  types  of  equipment  needed  for  pur- 
chase both  as  new  requirements  and  replacements. 

4.  Determine  department  and  individual  laboratory 
efficiencies. 

5.  Determine  training  requirements  for  department 
engineers  and  technicians  and  possible  user  training 
requirements. 

6.  Provide  decision  rules  for  equipment  disposition. 

7.  Determine  total  and  individual  laboratory  man- 
power requirements,  both  present  and  future. 

8.  Determine  calibration  intervals  from  statistical 
and  user  information. 

A  few  comments  on  calibration  intervals  seem  in 
order  here,  not  from  the  viewpoint  of  the  methods  to 
use  to  determine  correct  intervals  but  for  the  applica- 
tion of  interval  control.  Often  failure  rates  are  used 
to  determine  calibration  intervals  which  will  then  sat- 
isfy a  specified  quality  level.  Of  course,  this  process 
is  reversible  and  the  calibration  interval  can  be  used 
to  predict  the  failure  rate.  If  a  high  failure  rate  on 
certain  test  equipment  does  not  affect  the  quality  of 
the  end  product,  and  if  such  high  failure  rate  would 


optimize  the  total  company  cost  (i.e.,  user  cost  due  to 
the  high  failure  rate  versus  calibration  costs  at  shorter 
intervals),  then  by  all  means  such  equipment  should 
be  calibrated  at  intervals  allowing  the  higher  failure 
rate.  Government  specification  MIL-C-45662A  con- 
tains the  requirements  for  control  of  a  calibration  sys- 
tem for  Government  contractors.  This  military  specifi- 
cation states:  "All  measuring  and  test  equipment  ap- 
plicable to  the  contract  shall  be  subject  to  such  control 
as  is  necessary  to  assure  conformance  of  supplies  and 
services  to  contractual  requirements."  Therefore,  if 
the  contractor  can  show  complete  control  of  those  in- 
struments which  determine  conformance  of  supplies 
and  services  to  the  satisfaction  of  the  customer,  a 
higher  failure  rate  on  other  equipment  might  be  ap- 
propriate for  cost  optimizing. 

In  this  regard,  Convair  has  a  program  which  segre- 
gates critical  application  test  equipment  (CATE)  from 
all  other  test  equipment  in  the  company.  The  CATE 
equipment  is  then  placed  on  intervals  conforming  to 
customer  requirements  for  such  equipment.  The  rest 
of  the  equipment  is  placed  on  intervals  to  give  a  failure 
rate  which  management  has  determined  will  optimize 
company  cost.  Here  again,  the  segregation  of  CATE 
is  made  possible  by  visibility  of  the  equipment  and  the 
requirements  as  they  apply  to  both  Convair  and  the 
customer. 

The  computer  listing  which  is  supplied  to  manage- 
ment and  users,  identifying  the  users'  monthly  equip- 
ment costs,  is  a  powerful  tool.  Making  these  costs 
visible  to  the  user  and  to  higher  management  provides 
a  great  incentive  for  the  return  of  unnecessary  test 
equipment.  Infrequently  used  equipment  tends  to  as- 
sume storage  status  if  it  is  not  needed  at  calibration 
time.  These  costs  include  those  items  checked  out  of 
the  test  equipment  stockrooms  on  a  daily  basis.  The 
listing  provides  a  mechanism  to  tie  calibration  budget 
to  equipment  in  use. 

The  manpower-forecasting  tab  runs  can  show,  based 
on  present  intervals,  the  predicted  manpower  required 
for  items  due  for  calibration  in  any  future  time  period. 
The  runs  also  show  the  manpower  required  for  irreg- 
ular repair  work,  based  on  statistical  probabilities  com- 
piled from  the  past  history  of  presently  used  equipment. 

The  computerized  test  equipment  control  system  can 
provide  calibration  and  repair  targets,  based  on  statis- 
tical analysis  of  past  performance,  to  any  formula  de- 
sired by  management.  Summaries  of  the  individual, 
laboratory,  or  department  performance  against  those 
targets  for  any  time  period  can  be  provided  as  desired. 
Thus,  it  replaces  the  manual  computation  and  report- 
ing of  work  measurement. 

I  have  tried  to  highlight  some  points  of  visibility 
and  control  which  may  be  of  help  in  analyzing  anothe* 
company's  system.  The  basic  philosophy  is  that  if  one 
can  obtain  total  visibility  of  the  system  in  all  its  param- 
eters and  interfaces,  then  management  can  make  cor- 
rect decisions  and  decision  rules.  It  seems  obvious 
that  in  a  large  company  total  control  of  general-pur- 
pose test  equipment  should  be  a  function  of  the  Stand- 
ards and  Calibration  Laboratory. 

For  those  who  seek  a  "magic  formula"  I  suggest 
that  very  few  exist  in  actual  practice.    As  engineers, 
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we  have  a  great  propensity  for  formulas  because  we 
recognize  them  in  our  engineering  work  as  being  exact 
relationships  between  different  parameters.  In  man- 
agement, a  strict  formula  generally  means  that  a  com- 
promise has  been  made  in  the  name  of  uniformity  and 
often  at  high  cost.  This  is  not  to  advocate  that  man- 
agement textbook  formulas  are  no  good.  They  are 
very  good  for  their  intent,  which  is  mainly  to  give  in- 
sight into  the  relationships  of  various  operations  pa- 
rameters. Such  formulas  generally  presuppose  certain 
necessary  assumptions  which  preclude  their  being  ap- 
plied in  actual  practice.  Just  as  an  example,  consider 
the  development  of  the  most  efficient  system  of  reduc- 
ing a  calibration  backlog.  According  to  the  textbook 
theory  on  the  waiting-line  problem,  if  you  always  take 
those  items  first  which  have  the  shortest  calibration 
time,  then  you  will  have  the  most  efficient  system  in 
every  respect  except  one.  That  one  exception  is  that 
somebody  is  going  to  wait  an  awfully  long  time  for  his 
particular  equipment  to  get  calibrated.  Yet,  if  a  rule 
is  added  to  the  system  stating  that  no  unit  will  wait 
longer  than  a  specified  time,  the  efficiency  of  the  orig- 
inal system  is  reduced  by  up  to  100  percent  (in  com- 
parison to  a  first-come,  first-served,  system )  and  it  may 
no  longer  be  the  most  efficient  system.  How  these 
management  rules  and  practices  can  be  modified  to 
give  the  most  efficient  operation  depends  upon  each 
company's  particular  circumstances. 

Management  systems  should  be  as  flexible  as  possi- 
ble, yet  still  provide  the  necessary  control.  System 
feedback  should  impact  the  working  group  as  little  as 


possible,  just  as  we  strive  to  have  our  measuring  in- 
struments affect  the  basic  parameter  being  measured 
as  little  as  possible.  And  last,  but  certainly  not  least, 
the  system  should  be  designed  for  people. 

I  am  aware  that  figures  on  cost  reductions  achieved 
by  one  company,  even  in  the  form  of  percentages,  may 
not  mean  much  to  another  company.  This  is  under- 
standable since  there  is  no  way  to  compare  what  each 
was  formerly  doing  and  their  relative  efficiencies,  with 
what  they  are  presently  doing  and  their  relative  effi- 
ciencies. However,  as  a  matter  of  possible  interest, 
I  may  say  that  the  cost  savings  of  our  present  system 
over  the  previous  operational  method  are  approximate- 
ly $803,000  per  year.  This  saving  began  about  two 
years  after  we  started  gathering  data  for  analysis. 

Another  interesting  monetary  item  is  the  computer 
cost.  The  charges  for  computer  service  for  the  issu- 
ance of  all  work  forms  and  tab  runs  in  their  individual 
and  combined  variations  comes  to  $0.33  per  unit  of 
active  test  equipment  per  year.  This  charge  was  $0.40 
until  we  switched  from  a  card  system  to  a  tape  system. 
Tape  also  provides  much  greater  flexibility  and  faster 
service. 

Is  our  system  optimized  at  this  point?  We  don't 
think  so.  There  are  several  areas  in  which  we  believe 
we  can  make  improvements  to  our  present  system,  and 
we  are  working  on  them.  We  can  testify  that  the  pres- 
ent system  has  been  a  great  improvement  over  the 
previous  operation  and  has  led  us  to  additional  ideas 
for  future  changes. 
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DATA  SYSTEM  FOR  IMPROVING  INSTRUMENT  RELIABILITY 

Frank  J.  Dyce 

Metrology  Laboratory,  Martin  Marietta  Corporation,  Orlando,  Fla.  32805 

This  paper  discusses  the  importance  of  monitoring  and  control  of  instrument  reliability.  It  suggests 
that  a  minimum  reliability  level  be  set  as  a  goal.  It  discusses  the  interval  adjustment  scheme  used  at 
Martin  Marietta  wherein  individual  instrument  performance  determines  calibration  interval  as  well 
as  repair  and  surplus.  A  thorough  explanation  is  given  of  the  computerized  data  system  used  at 
Martin  Marietta  to  monitor  instrument  reliability  and  to  isolate  repetitive  problem  areas.  Several 
examples  are  given  of  reliability  problems  which  were  solved  through  use  of  the  system. 

A  unique  standardization  committee  is  discussed  which  justifies  the  company's  sole-source  purchase 
of  general-purpose,  reliable  electronic  test  equipment. 

The  paper  concludes  that  striving  towards  a  reliability  goal  will  result  in  improved  reliability  and 
reduced  cost. 

Key  words:  Calibration  data  system;  calibration  interval  adjustment  method;  instrument  reliability; 
instrument  standardization. 


1.  Introduction 

Instrument  reliability  is  a  subject  of  much  concern 
to  metrologists  today.  The  95  percent  reliability  re- 
quirement implied  by  Military  Handbook  52  has  caused 
some  laboratories  to  develop  elaborate  mathematical 
formulations  and  contrived  definitions  of  typical  in- 
strument reliability  to  meet  it.  These  manipulations 
were  initially  undertaken  to  meet  the  implied  specifi- 
cation. Now  we  are  coming  of  age  and  can  better  ap- 
preciate the  importance  of  reliability.  We  know  it  is 
impossible  to  achieve  100  percent  reliability ;  however, 
there  is  a  definite  reason  to  strive  for  an  optimum  level. 
This  optimum  level  is  dependent  on  the  laboratories' 
resources  and  capabilities.  It  should  be  a  published 
goal  capable  of  measurement. 

Reliable  equipment: 

1.  can  make  reliable  measurements; 

2.  needs  to  be  calibrated  less  frequently; 

3.  requires  less  repair  time; 

4.  requires  fewer  replacement  parts; 

5.  has  a  higher  utilization  rate. 

Probably  the  major  reason  for  having  reliable  equip- 
ment is  that  most  instrument  users  assume  instruments 
are  absolutely  accurate.  To  protect  them  and  the  prod- 
uct, instruments  should  be  made  as  reliable  as  feasible. 

The  disadvantages  of  high  reliability  are: 

1.  Calibration  intervals  might  be  shorter. 

2.  Utilization  rate  might  be  lower. 

3.  Calibration  time  might  be  longer. 

We  have  made  95  percent  reliability  our  goal  and 
we  are  getting  close  to  it.  We  measure  reliability  in  a 
very  basic  way:  number  of  instruments  received  for 
calibration  which  are  in-tolerance  ( with  respect  to  pub- 


lished specification )  versus  number  of  instruments 
calibrated. 

Our  laboratory  strives  for  this  goal,  using  two  basic 
approaches:  an  interval  adjustment  scheme  based  on 
individual  instrument  reliability;  and  a  minimum-in- 
put automatic  data  processing  system. 

2.  Interval  Adjustment  Scheme 

The  initial  calibration  interval  is  based  on  the  av- 
erage interval  for  instruments  of  the  same  manufac- 
turing model  number  or  on  experience  with  instru- 
ments of  the  same  type.  Each  time  an  instrument  is 
calibrated  its  past  calibration  history  accompanies  it 
(fig.  1).  The  calibration  technician  examines  the  his- 
tory and  the  condition  of  the  equipment  to  determine 
its  next  calibration  interval  as  well  as  to  determine 
repetitive  failures.  If  the  instrument  is  out  of  toler- 
ance, the  interval  is  reduced  20  percent;  if  it  is  in  tol- 
erance three  consecutive  times,  the  interval  is  increased 
20  percent  for  each  time  thereafter.  (The  intervals 
vary  in  the  following  steps:  30,  40,  50,  60,  70,  90.  110, 
130,  160,  190,  225,  270,  320,  365  days).  Thus,  our 
major  effort  is  expended  on  unreliable  equipment  and 
our  workload  is  balanced,  since  an  instrument  never 
out-of-tolerance  is  calibrated  only  once  a  year.  When 
two  consecutive  out -of -tolerances  occur,  repair  is 
mandatory;  adjustment  is  obviously  inadequate.  The 
faulty  components  must  be  replaced  and  the  instrument 
checked  for  stability  for  two  days.  When  three  con- 
secutive out-of-tolerances  occur,  the  instrument  is  con- 
sidered for  surplus.  Figure  2  shows  that  by  using 
this  system,  our  lab  is  now  doing  fewer  actual  cali- 
brations (solid  line)  on  twice  as  many  pieces  of  equip- 
ment (dashed  line)  as  we  were  eight  years  ago;  we 
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Figure  1.  Calibration  history  and  transmittal. 
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equipment  inventory  (clashed  line). 


have  improved  reliability  and  lowered  costs  simultane- 
ously. Our  current  reliability.  90.3  percent,  is  9  per- 
cent higher  than  our  total  past  reliability  and  is  stead- 
ily increasing. 

3.  Automatic  Data  Processing  System 

A  calibration  data  system  is  required  to  indicate 
the  past  condition  of  the  equipment,  to  determine  the 
need  for  interval  adjustments,  and  to  meet  inventory 
and  Mil  Spec  requirements.  Our  data  system  was  de- 
signed to  collect  only  data  that  we  would  actually  use. 
The  input  data  transmittal  (fig.  1)  is  designed  for 
minimum  data  handling  and  minimum  chance  for  er- 
ror.    For  an  in-tolerance  instrument,  the  technician 


enters  the  following  data:  equipment  identification 
number,  condition  as  received  (I  in  tolerance,  0  out 
of  tolerance),  calibration  date,  recalibration  due  date, 
calibration  interval,  stamp  number,  and  calibration 
hours.  For  an  out-of-tolerance  instrument,  he  adds  the 
repair  hours,  replacement-part  cost,  and  coded  defect. 
The  coded  defect  is  the  only  complicated  part  of  the 
system.  Each  technician  has  a  mnemonic  code  book 
with  codes  listed  for  the  family  of  instruments  he  re- 
pairs. The  technician  codes  each  system  failure;  What 
was  wrong,  how  it  was  wrong,  what  part  caused  the 
failure,  and  what  was  done  to  the  part.  For  example, 
if  a  signal  generator  had  low  output  and  vacuum  tube 
V6  was  replaced  to  correct  it,  it  would  be  coded 
0LV6R;  which  reads  Output  (0)  Low  (L)  Vacuum 
Tube  (V)  Circuit  Symbol  6  Replaced  (R).  The  dis- 
advantage that  the  coding  complicates  the  data  trans- 
mittal is  offset  by  the  data's  availability  for  later  com- 
puter analysis. 

We  have  40  families  of  instruments  for  which  we 
record  discrete  parameter  data.  There  are  many  phi- 
losophies on  how  much  data  to  collect.  Our  philosophy 
is  that  it  is  pointless  to  collect  data  on  reliable  instru- 
ments; therefore  we  have  established  the  10  most  criti- 
cal parameters  on  our  most  unreliable  families  of  in- 
struments. These  parameters  and  their  upper  and 
lower  tolerance  limits  are  inserted  in  the  calibration 
recall  history  header  (fig.  3).  Each  time  an  instru- 
ment is  calibrated,  the  technician  records  the  param- 
eters prior  to  adjustment  or  repair.  He  compares  the 
parameters  against  the  limits  and  the  previous  read- 
ings to  determine  the  instrument's  drift  characteristics 
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Figure  3.  Parameter  history. 


and  the  necessity  for  adjustment.  We  have  a  directive 
which  dictates  that  if  a  measurement  is  within  half  its 
tolerance  limit,  leave  it  alone.  If  it  is  outside  half  its 
limit,  adjust  it  to  nominal.  This  permits  adjustments 
to  settle  at  a  stable  value.  The  computer  analyzes  this 
discrete  data  on  a  family  basis  and  produces  a  report 
which  we  use  to  pinpoint  instrument  problem  areas 
and  to  eliminate  unnecessary  steps  in  the  calibration 
procedure.  Both  Mil  Spec  and  customer  requirements 
require  us  to  maintain  a  data  system  for  recall.  The 
computer  does  this  with  a  simple  inventory  technique; 
however,  we  feel  that  the  systems  that  perform  recall 
alone  fail  to  make  full  use  of  the  computer's  potential. 
We  use  the  computer  for  inventory,  recall,  history  and 
records,  and,  most  importantly,  special  reports.  We 
have  several  reports  which  are  used  to  improve  re- 
liability; the  first  is  the  Reliability  Summary  Report 
described  below. 

4.  Reliability  Summary  Report 

This  report  is  our  measurement  of  reliability.  It  is 
required  since,  in  order  to  strive  towards  our  goal,  we 
must  have  a  measure  of  performance.  This  report 
(fig.  4)  lists  reliability  by  class,  manufacturer,  and 
model  number.  It  tells  us  first  how  reliable  we  are, 
and  it  pinpoints  problem  areas  and  isolates  them  to 
classes  of  instruments,  manufacturers,  and  model  num- 
bers. The  report  is  examined  monthly  to  isolate  our 
least  reliable  major  families  of  instruments,  the  fam- 
ilies which  are  causing  our  reliability  to  be  less  than 
95  percent.  It  also  lists  the  family  reliability  based 
on  the  last  calibration,  to  indicate  what  our  current 


reliability  is.  Monitoring  this  report,  we  noted  that 
the  current  reliability  of  a  family  of  38  function  gen- 
erators had  decreased  from  its  overall  level  of  89  per- 
cent to  a  reliability  of  66  percent  over  a  period  of 
several  months.  An  investigation  revealed  the  major 
cause  of  failure  was  signal  distortion  which  required 
adjustment.  A  further  check  revealed  the  distortion 
analyzer  used  in  the  calibration  was  defective,  causing 
the  generators  to  be  adjusted  to  an  order  of  magnitude 
less  distortion  than  what  they  actually  required.  The 
analyzer  was  repaired  and  the  current  reliability  of  the 
function  generators  has  improved  to  99  percent. 

The  summary  report  indicated  one  family  of  44  rf 
signal  generators  had  an  historical  reliability  of  66 
percent  and  a  current  reliability  of  64  percent.  This 
indicated  not  only  that  the  reliability  was  bad,  but 
that  it  was  getting  worse.  The  Quarterly  Reliability 
Report  had  to  be  consulted  to  determine  if  we  had  an 
individual  instrument  problem  or  a  family  problem. 

5.  Quarterly  Reliability  Report 

The  Quarterly  Reliability  Report  has  the  same  basic 
information  and  format  as  the  summary  report  but  it 
includes  each  individual  instrument's  reliability  by 
identification  number.  This  report  (see  fig.  5)  indi- 
cated the  individual  instruments'  reliabilities  varied 
between  54  and  88  percent  with  an  overall  reliability 
of  64  percent.  It  is  difficult  to  prove  what  causes  in- 
struments to  fail — whether  it  is  misuse,  mishandling, 
poor  design,  etc.;  however,  if  all  instruments  of  the 
same  type  have  poor  reliability,  the  problem  is  likely 
to  be  endemic  in  the  type,  and  that  is  what  should  be 
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Figure  4.  Reliability  Summary  Report. 
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Figure  5.  Quarterly  Reliability  Report. 


investigated.  The  problem  in  our  example  appeared 
to  be  a  family  problem  and  it  was  examined  accord- 


ingly. The  investigation  required  the  use  of  another 
report,  the  Trouble-Shooting  Summary  Report. 


6.  Trouble-Shooting  Summary  Report 

The  Trouble-Shooting  Summary  Report  summarizes 
all  instrument  repair  information  by  manufacturing 
model  number.  It  sorts  the  repair  information  and 
lists  it  in  alphanumeric  sequence,  with  like  codes  to- 
taled by  manufacturing  model  number. 

The  family  of  signal  generators  we  investigated  had 
a  repetitive  problem  (see  fig.  6).  The  problems  were 
0LV8  out  of  tolerance  40  times  and  OLV6  out  of  tol- 
erance 30  times,  both  requiring  corrective  action  of  R. 
Using  the  mnemonic  code  book,  these  failures  are  de- 
coded as  Output  Low  Vacuum  Tube  8  replaced  40 
times  and  Output  Low  Vacuum  Tube  6  replaced  30 
times;  i.e.,  tubes  V6  and  V8  were  replaced  70  times 
on  44  generators  in  one  year.  This  represented  50  per- 
cent of  the  problems  with  this  family  of  instruments. 
A  further  investigation  of  the  individual  histories  re- 
vealed that  V6  and  V8  had  been  replaced  in  several 
instruments  on  consecutive  calibrations.  These  instru- 
ments were  recalled  and  an  examination  revealed  that 
supply  voltages  to  V6  and  V8  were  off  by  over  20 
percent.  This  caused  the  tubes  to  operate  in  an  over- 
loaded condition  with  reduced  output  and  a  very  short 
life.  Identical  conditions  were  found  on  4  out  of  the 
5  generators  recalled;  the  cause  of  the  difficulty  was 
that  technicians  inadvisedly  replaced  weak  output  tubes 
when  the  trouble  was  improper  supply  voltages.  The 
instruments  were  in  tolerance  when  they  left  the  lab 
but  the  tubes  rapidly  deteriorated  so  that  the  output 
dropped  off.  The  technicians  were  informed  of  the 
trouble,  an  instruction  to  check  all  supply  voltages  was 
added  to  the  calibration  procedure,  and  a  filament 
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Figure  6.  Troubleshooting  Summary  Report. 


voltage  parameter  was  added  to  the  calibration  trans- 
mittal. Using  these  steps,  we  improved  the  reliability 
from  66  percent  to  the  current  97  percent.  The  data 
were  monitored  to  determine  how  often  low  output 
occurred  and  to  note  the  drift  in  filament  voltage. 
They  both  appeared  stable  after  the  corrective  action. 

7.  Surplus  Selection 

The  reliability  report  is  also  used  to  segregate  in- 
struments for  surplus.  An  arbitrary  value  of  50  per- 
cent reliability  was  selected  and  a  report  listed  all  in- 
struments below  that  value.  Further  study  was  then 
conducted,  utilizing  cost  information  previously  stored 
in  the  computer,  to  make  the  surplus  decision.  In 
this  way  we  eliminated  our  most  unreliable  instruments 
which  had  the  highest  maintenance  costs.  We  have  a 
set  rule  that  no  instrument  less  than  50  percent  reliable 
may  be  used  for  acceptance  testing;  we  will  not  cer- 
tify it.  This  accelerates  an  otherwise  slow  process  of 
designating  equipment  surplus. 

8.  Manpower  Adjustment 

We  have  been  able  to  reduce  manpower  by  exam- 
ining calibration  intervals  versus  reliability.  We  ex- 
tended intervals  on  with  good  reliability  until  they 
equalled  those  for  older  instruments  of  the  same  type 
with  similar  reliability.  We  also  reduced  intervals  on 
instruments  with  poor  reliability.  This  has  represented 
one-year  savings  of  3,000  hours  calibration  time  on 
8,000  instruments.  The  method  of  extension  was  fa- 
cilitated by  manipulating  the  computer  printout.  An 
interval  change  code  (CI50)  was  attached  to  the  re- 
call history  (fig.  7),  so  that  when  the  instrument  came 
due  for  calibration,  the  technician  noted  the  CI50, 
changed  the  interval  to  50  days,  and  removed  the 
change  code.  In  this  way  we  did  not  have  to  locate 
the  instrument  physically  or  depend  on  a  checklist. 
The  basic  interval  adjustment  was  derived  from  an 
algebraic  sum  of  the  conditions  as  received;  e.g., 
II0II0IIII  would  yield  zero  net  interval  increase  and 
the  interval  would  remain  at  90  days,  say,  whereas 
IIIIIIII00  would  yield  a  net  increase  of  80  percent 
and  an  interval  of  190  days.  Even  though  both  instru- 
ments are  equally  reliable,  their  intervals  will  differ 
by  a  factor  of  two. 

9.  Standardization  Committee 

Our  company  has  an  Equipment  Standardization 
Committee  made  up  of  representatives  from  Engineer- 
ing, Manufacturing,  Procurement,  Facilities  and  Me- 
trology. The  Committee's  purpose  is  to  generate  a 
Standardization  List  of  equipment  recommended  for 
purchase.  Before  any  general-purpose  electronic  test 
equipment  can  be  purchased,  it  must  be  on  our  Stand- 
ardization List.  The  committee  meets  bi-weekly  and 
prepares  a  comparison  matrix  to  evaluate  equipment 
specifications,  performance,  reliability,  and  price.  A 
matrix  is  made  for  each  general  item — oscilloscopes, 
counters,  digital  voltmeters,  etc.  Once  selected  by  the 
Committee,  an  item  must  be  evalauted  by  the  Metrol- 
ogy Laboratory  before  being  included  on  the  list.  In 
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this  way  we  ensure  that  it  meets  its  advertised  specifica- 
tions and  is  serviceable.  Standardizing  on  test  equip- 
ment has  many  advantages: 

1.  Operator  familiarity  with  a  single  item. 

2.  Quantity  discounts. 

3.  One  set  of  spare  parts. 

4.  More  efficient  calibration  and  maintenance. 

5.  Purchase  of  the  best  instrument  for  the  job. 
Previous  vendor  history  records  are  consulted  through 
the  Family  Summary  Report  to  insure  that  calibration 
costs  are  minimized  and  reliability  is  optimized. 

10.  Family  Summary  Report 

The  Family  Summary  Report  indicates  by  family: 

1.  Average  calibration  hours. 

2.  Average  repair  hours. 

3.  Average  repair  parts. 

4.  More  efficient  calibration  and  maintenance. 

5.  Average  calibration  interval. 

With  this  type  of  information  available,  we  can 
readily  forecast  the  time  required  to  perform  a  cali- 
bration on  any  instrument.  We  can  isolate  the  instru- 
ments on  which  we  are  spending  excessive  amounts  of 
time.  We  can  compare  relative  performance  of  like 
instruments.  For  example,  the  report  listed  the  fol- 
lowing data  on  two  equivalent  models  of  differential 
voltmeters. 


Mfg.& 
Model 
No.  Cost 


No.  of  No.  Avg.  Rep.  Avg.  Cat.  Avg.  Avg.  Cal. 
Instr.  Calib.  Hrs/Cal.  Hrs/Cal.  Rel.  Interval 


Brand  X  $1240 
Brand  Y  $1275 


27 
21 


0.18 
1.11 


2.11 
3.18 


57 


166 

85 


The  Brand  X  instrument  is  more  reliable,  requires  con- 
siderably less  calibration  and  repair  time,  and  can  be 
calibrated  half  as  often. 

Thus,  even  though  instruments  might  be  equivalent 
in  specification  and  purchase  price,  their  actual  his- 
tory can  reveal  cost  differences  of  great  significance. 
This  is  the  area  we  try  to  emphasize  in  our  Standardi- 
zation Committee  meetings. 

11.  Conclusion 

Although  the  95  percent  reliability  statement  of  Mili- 
tary Handbook  52  is  often  criticized,  it  does  serve  a 
worthwhile  purpose.  Striving  for  this  goal,  we  are  pres- 
ently increasing  our  instrument  reliability,  reducing 
our  operating  costs,  and  feel  confident  we  are  improv- 
ing product  reliability.  The  day  is  not  too  far  off  when 
mean  time  between  failures  will  be  specified  on  preci- 
sion test  equipment.  Even  some  of  the  best-known  in- 
strument manufacturers  sell  an  unreliable  product. 
Metrology  labs  throughout  the  country,  preferably 
through  the  auspices  of  NCSL,  should  band  together 
to  collect  similar  reliability  data  and  should  urge  man- 
ufacturers to  produce  more  reliable  instruments. 
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NEW  U.S.  AIR  FORCE  AUDIT  PROGRAM 
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Metrology  Center  (AFLC),  Newark  Air  Force  Station,  Newark,  Ohio  43055 

This  paper  describes  a  unique  quality  assurance  audit  program  used  for  testing  worldwide  Precision 
Measurement  Equipment  Laboratories  (PMEL's).  Conventional  methods  utilized  during  previous 
audits  are  examined  briefly  and  disadvantages  noted.  The  advent  of  highly  sophisticated  systems  and 
test  equipment  has  placed  increased  emphasis  on  the  experience  and  skill  level  of  the  technicians 
who  provide  calibration  support  at  PMEL's.  Specific  configuration  of  a  unique  package  designed  to 
evaluate  this  skill  level  is  discussed  in  some  detail  and  results  obtained  during  the  past  year  are 
presented. 

Key  words:  Audit  examinations;  audit  package;  Precision  Measurement  Equipment  Laboratories 
(PMEL)  ;  technician  skills;  U.S.  Air  Force. 


1.  Introduction 

The  United  States  Air  Force  operates  a  large  num- 
ber of  Precision  Measurement  Equipment  Laboratories 
(PMEL's)  strategically  located  throughout  the  world. 
These  are  audited  by  five  Air  Materiel  Area  Labora- 
tories (AMA  Labs)  located  within  the  continental 
United  States.  Technical  direction  and  management 
of  the  Air  Force  Calibration  Program  is  the  responsi- 
bility of  the  Air  Force  Logistics  Command  (AFLC) 
and  has  been  delegated  to  the  Aerospace  Guidance  and 
Metrology  Center  (AGMC)  at  Newark  Air  Force  Sta- 
tion in  Ohio.  Each  base  PMEL  is  audited  on  an  an- 
nual basis  by  its  AMA  with  the  latter  being  certified 
by  AGMC.  In  general,  examination  is  concentrated  in 
five  general  areas — 

1.  Environment:  temperature,  humidity,  dust, 
housekeeping. 

2.  Software:  Technical  Orders  (T.O.'s) , reference 
library. 

3.  Reference  standards:  traceable  to  NBS  or  other 
laboratories. 

4.  Test  equipment:  signal  generators,  'scopes, 
multimeters,  etc. 

5.  Personnel:  adequacy  of  training,  experience, 
and  skill. 

In  addition,  adequacy  of  facilities  and  administrative 
methods  is  thoroughly  investigated. 

2.  Previous  Audit  Method 

Past  audits  have  utilized  basic  standards  in  the  elec- 
trical and  mechanical  areas  whose  absolute  values 
could  easily  be  predicted  by  a  laboratory  technician 
to  be  within  the  tolerances  allowed  without  actually 


measuring  those  values.  This  was  true  because  the 
nominal  values  of  the  selected  standards  are  known 
because  of  the  model  or  part  number.  For  example, 
a  GR  model  1482-L  inductor  has  a  nominal  value  of 
100  mil,  adjusted  to  be  within  0.1  percent  of  exact 
value.  Without  making  any  measurements  whatever, 
a  lab  technician  could  report  the  nominal  value  and 
be  graded  as  acceptable  if  the  tolerance  allowed  on  his 
measurement  were  0.1  percent.  As  another  example, 
suppose  he  were  asked  to  measure  a  high-quality  gage 
block  whose  nominal  value  is  known.  He  might  re- 
port the  exact  nominal  value  (or  even  change  it  slightly 
to  make  the  actual  measurement  more  palatable),  and 
if  the  allowed  tolerance  were  ±  10  fiin,  he  would  pass 
with  flying  colors! 

3.  Proposed  Revisions 

Two  possible  solutions  were  proposed  to  avoid  these 
situations.  Allowable  tolerances  could  be  reduced  or 
the  standard  item  could  be  changed  in  value.  Re- 
duced tolerances  were  found  to  be  unrealistic  because 
of  shipping  shocks  or  even  hidden  damage  not  under 
control.  Stability  of  some  standards  was  found  to  be 
poor  because  of  hysteresis  effects  caused  by  tempera- 
ture excursions  experienced  in  transit.  A  second  solu- 
tion, that  of  supplying  a  measurement  problem  which 
could  not  be  "guesstimated,"  was  also  considered.  For 
example,  one  surface  of  an  optical  flat  could  be  re- 
figured  to  produce  a  slight  cylinder  or  sphere  and  the 
lab  technician  required  to  define  the  measured  sur- 
face accurately.  A  standard  resistor  or  capacitor 
could  be  "fudged"  to  have  an  actual  value  on  the  order 
of  a  few  hundred  ppm  away  from  nominal.  Unless 
the  allowable  tolerances  were  increased,  however,  the 
same  obvious  deviations  might  be  attributed  to  ship- 
ping or  local  handling. 
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4.  A  INew  Approach 

A  unique  approach  to  laboratory  audits  was  envi- 
sioned as  early  as  1060.  The  number  of  possible  prob- 
lems has  grown  over  the  years  from  a  half  dozen  to 
more  than  20,  about  equally  divided  between  electrical 
and  mechanical  areas.  New  devices  are  being  devel- 
oped so  that  completely  different  problems  may  be  util- 
ized during  successive  audits.  Sources  of  supply  for 
useful  items  include  toy  shops,  surplus  electronic  out- 
lets, "do-it-yourself"  building  supplies,  and  scientific 
houses. 

Problems  are  designed  to  test  the  ability  of  the  lab- 
oratory technicians  to  work  together  as  a  team  in  the 
solution  of  the  test  problems.  Emphasis  is  placed  on 
comprehension,  original  thinking,  and  measurement 
techniques  rather  than  on  resolution  and  accuracy  to 
parts  per  million.  The  device  being  measured  may  be 
sensitive  to  its  environment  or  to  the  pattern  of  the 
applied  stimulus.  Thus,  test  results  may  vary  from 
one  laboratory  to  another  because  the  auditor  may 
change  the  specified  test  parameters,  resulting  in  many 
different  answers  to  a  particular  problem — and  all 
can  be  correct.  Some  problems  may  specify  the 
choice  of  test  equipment;  others  allow  a  very  wide 
selection. 


5.  Audit  Implementation 

Problem  layout  and  nominal  parameters  are  sent  to 
the  audited  laboratory  at  least  one  month  in  advance 
of  the  appearance  of  the  auditor  and  actual  test  pack- 
age so  that  they  may  be  studied  and  measurement 
schemes  prepared  for  the  auditor's  approval.  One 
problem  is  designated  as  required,  with  an  additional 
three  of  the  remaining  six  to  selected  by  the  audited 
laboratory.  Most  problems  may  be  solved  in  several 
different  ways,  but  any  scheme  proposed  must  be  ac- 
ceptable to  the  auditor  before  being  implemented. 
Assistance  may  be  provided  when  required  and,  at 
the  conclusion  of  the  test,  other  schemes  having  equal  or 
greater  merit  may  be  discussed.  A  valid  error  analysis 
must  be  included  with  each  final  answer  produced. 

All  schemes,  test  data,  compilation  of  results,  and 
error  analyses  are  reported  in  triplicate;  one  copy  re- 
mains at  the  tested  site,  one  is  filed  at  the  AMA,  and 
one  is  sent  to  AGMC  for  analysis.  When  such  analy- 
sis indicates  lack  of  suitable  test  equipment,  reference 
standards,  or  training,  limited  certification  may  be  im- 
posed or  recommendations  made  for  future  augmenta- 
tion of  the  laboratory  capabilities. 

To  facilitate  uniform  reporting,  a  check  list  is  used 
for  each  separate  problem  as  follows: 


Audit  Package  Checklist 

NOTE:  Minimum  requirements — problem  2b  plus  a  choice  of  three  additional  problems.  Data  sheets  and  measure- 
ment schemes  are  to  be  prepared  in  triplicate  by  audited  laboratory. 

Name  of  auditor/Symbol/Date  

PMEL/AMA  Symbol,  Building  

Problem  number   

Work  location  illumination  level,  foot  candles   

temperature  limits,  °C   

humidity  limits,  %  RH   

Yes  No 

1.  Was  measurement  scheme  prepared  in  advance?   

2.  Is  it  acceptable  to  auditor?  (If  not,  on-spot  OJT  may  be  provided  by  auditor  or   

alternate  problem  may  be  selected). 

3.  Are  current  AFTO  108's  affixed  to  the  test  equipment?   

4.  Were  model  and  serial  numbers  of  equipment  recorded   

5.  Were  model  temperature/humidity  recorded?   

6.  If  desk  calculator,  reference  library,  or  special  equipment  were  needed,  were  ihey 

available?  (Explain  with  attachment,  if  necessary).   

7.  If  Ac  ratio  measurements  were  made,  were  proper  corrections  made  for  zero  and   

unity  balances? 

8.  Were  shielded  cables  used  (if  necessary)?   

9.  On  dc  ratio  measurements,  were  proper  lead  compensation  corrections  made  or  

analyzed  for  possible  error? 

10.  Was  effort  wasted  in  attempting  greater  resolution  of  accuracy  than  required  by  

the-  problem? 

11.  Were  loading  errors  considered?   

12.  Was  more  than  one  method  used  to  solve  problem?   

General  comments  as  to  capability  of  audited  personnel,  attitude,  cooperation,  motivation,  etc.  (Use  reverse  side). 
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6.  Audit  Package  Configuration 

The  actual  package  for  the  electrical  audit  is  de- 
signed to  test  proficiency  in  the  use  of  an  Electrical 
Measurement  Console  ( EMC )  issued  to  all  base  labo- 
ratories. Two  standard  "mini-boxes"  containing  all 
components  are  housed  in  a  small,  foam-padded  satchel. 


Figures  1.  2,  and  3  show  the  method  of  packaging; 
figures  4  and  5  the  component  layout. 

Many  variations  are  possible  for  the  oscillator  cir- 
cuit board  depending  upon  the  degree  of  sophistica- 
tion desired.  Two  representative  schematics  are  shown 
in  figures  6  and  7. 


Figure  3.  Chassis  layout  "B"  items. 


Figure  1.  Portable  audit  package. 
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OSCILLATOR  CIRCUIT 
Figure  6.  Typical  oscillator  schematic  diagram. 
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SPECIAL  SERIES 
Figure  7.  Special  series  oscillator  schematic. 


7.  Audit  Problems  (Instructions 
Accompanying  Package) 

1.  Caution.  Avoid  rough  or  prolonged  handling  of  the 
metal  boxes — all  internal  components  are  temperature- 
and  shock-sensitive  to  some  degree.  Some  components 
can  be  damaged  by  application  of  excessive  potentials  or 
currents  so  that  instruction  should  be  followed  exactly. 

2.  All  serial  numbers  ending  in  "A"  contain  a  com- 
plex signal  source  available  at  BNC  connector,  terminal 
#6,  and  a  zener  diode  of  nominal  6.8  Vdc  breakdown 
between  terminals  #3(  +  )  and  #4(  — ).  Note  that  ter- 
minals #5  is  case  ground. 

a.  zener  diode  problem.  Avoid  handling  the 
metal  boxes  unnecessarily.  Determine  the  voltage  drop 
across  terminals  3-4  when  a  specified  current  between 
1  and  5  mA  is  made  to  flow  through  the  diode  in  such 
a  direction  as  to  make  terminal  3  positive.  Using 
only  components  of  the  Electrical  Measurement  Con- 
sole (EMC),  devise  a  scheme  for  measurement. 

Do  not  use  a  grounded  current  source  and  do  not 
use  terminal  5  for  this  test.  Scheme  must  make  it 
impossible  to  send  a  current  through  the  diode  or  as- 
sociated circuitry  greater  than  5  mA.  Scheme  must 
be  approved  by  the  auditor  and  all  current  and  po- 
tential measurements  must  have  an  uncertainty  less 
than  0.01  percent.  Final  report  must  detail  the  scheme 
used  (schematic  diagram)  and  include  an  error  analy- 
sis to  justify  the  0.01  percent  uncertainty.  If  the  0.01 
percent  uncertainty  cannot  be  attained,  a  full  explana- 
tion must  be  provided  with  a  statement  as  to  what  un- 
certainty can  be  realized.  , 

b.  complex  signal  problem/.  Avoid  unnecessary 
handling  of  the  metal  box.  Apply  10  V  dc  to  termi- 
nals 3(  +  )  and  5(— )  from  a  source  which  is  current 
limited  to  5  mA.  Applied  voltage  must  have  a  mag- 
nitude uncertainty  less  than  0.1  Jpercent.  A  complex 
signal  should  now  be  found  at  BNC  connector,  termi- 
nal #6.  No  restrictions  are  placed  on  your  choice  of 
test  equipment.  Determine  and, 'record  all  parameters 
of  the  complex  signal  and  document  the  method  of 
test.  Include  an  error  analysis  statement  of  estimated 
uncertainties.  f 

3.  All  serial  numbers  ending  in  "B"  contain  the 
following:  . 

a.  Terminals  7—8,  nominaj  0.1  /xF 

b.  Terminals  8—9,  nominal  3500  fi 

c.  Terminals  10-11—12,  tapped  toroidal  inductor 

d.  Terminals  13-14,  capacitor,  greater  than  1  /aF 

e.  Terminals  16—17,  nominal  1  /iF 

f.  Terminals  11—15,  thermistor,  approximately  50 
kft 

Note  that  terminal  #15  is  case  ground. 

4.  a.  phase  defect  problem.  Using  the  Phase  Stand- 
ard, model  311,  and  the  necessary  ancillary  equipment 
of  your  choice,  drive  terminals  7  and  9  with  10  V  rms 
at  1  kHz  and  measure  the  phase  angle  of  the  output 
terminal  8  with  reference  to  the  input  drive  voltage. 
Now  reverse  the  drive  and  again  measure  the  output 
phase  angle.  It  will  be  found  that  the  sum  of  these 
two  measured  angles  is  slightly  less  than  90  degrees 
(if  the  experiment  is  correptly  performed).  This  ap- 
parent "phase  defect"  is  claused  by  an  imperfect  ca- 
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pacitor,  i.e.,  one  that  has  a  dissipation  factor  greater 
than  zero.    From  the  phase  angle  "defect"  noted,  cal- 

i  culate  the  true  dissipation  factor  of  the  capacitor,  be- 
tween terminals  7-8.    Now,  using  the  EMC  Capaci- 

!  tance  Bridge,  model  707B,  and  AC  Generator-Detec- 
tor, model  861A,  measure  the  same  capacitor  and  com- 
pare results.  Account  for  any  disagreement  in  "D." 
Measure  the  resistance  between  terminals  8—9  and, 
using  the  measured  capacitance  between  terminals  7—8, 
calculate  the  output  phase  angle  relative  to  the  input 

|  terminals  7—9  and  account  for  any  difference  over  the 
measured  value  obtained  using  the  model  311. 

b.  CORE  saturation  problem.  Using  your  choice 
of  test  equipment,  devise  a  scheme  for  determining 
the  maximum  rms  voltage  that  can  be  applied  to  the 
inductor  terminals  10—12  before  core  saturation  occurs. 
Express  this  maximum  voltage  (V,  rms)  as  Kf  where 
K  is  a  constant  (to  be  determined)  and  /  is  the  ap- 
plied frequency  in  hertz. 

c.  AC  RATIO  PROBLEM.  Using  only  components  of 
the  EMC,  devise  a  scheme  to  measure  the  turns  ratio 
between  terminals  11-12  compared  to  terminals  10—12. 
This  should  be  done  using  a  drive  voltage  and  fre- 
quency as  specified  by  the  auditor.  Determine  also 
the  phase  angle  between  the  voltage  on  terminals  11—12 
as  compared  to  the  drive  voltage  on  terminals  10—12. 

d.  thermistor  problem.  Using  the  model  242 
resistance-measuring  system,  make  a  two-terminal  re- 
sistance measurement  between  terminals  8—15,  using 
the  minimum  applied  voltage  needed  to  obtain  a  re- 
sistance measurement  with  a  resolution  of  0.1  percent. 
Note  that  terminal  15  is  case  grounded.  Avoid  han- 
dling the  metal  box  unnecessarily.  Record  this  meas- 
ured value  and  the  average  room  temperature  near 
the  metal  box  at  the  time.  Note  the  excitation  voltage 
applied  to  the  system  and  calculate  the  voltage  appear- 
ing across  and  the  power  dissipated  in  the  unknown 
thermistor.    Record  all  values. 

Hint:  It  may  be  helpful  to  measure  this  resistor  while 
it  is  protected  from  room  temperature  changes.  To 
do  so,  form  two  3-ft  lengths  of  No.  22  AWG  insulated 
copper  wire  (telephone  wire)  into  a  tightly  twisted 
pair  and  guide  it  through  the  small  hole  in  the  right 
side  of  the  carrying  case  so  that  the  metal  box  (S/N 
XXX  B)  can  be  completely  covered  in  its  polyfoam- 
insulated  container,  with  the  cover  closed  during  meas- 
urement. Connect  the  metal  trim  on  the  carrying 
case  to  ground  to  eliminate  hand  capacity  effects.  Now 
determine  the  voltage  that  must  be  applied  to  the  test 
item  to  cause  sufficient  self-heating  of  the  bead  ther- 
mistor to  change  the  measured  resistance  value  by  ap- 
proximately 20  percent.  Explain  the  result  noted. 
Record  all  data. 

e.  CAPACITANCE  AND  DISSIPATION  FACTOR  PROBLEM. 

Using  only  the  models  707B  and  816A  and  any  com- 
ponents found  in  S/N  XXX  B,  devise  a  scheme  to 
measure  the  capacitance  and  dissipation  factor  of  the 
capacitor  connected  to  terminals  13—14.  Record  all 
data  used  and  have  auditor  check  your  results  for  ac- 
curacy. State  expected  accuracy  to  be  obtained  for 
both  "C"  and  "D."  The  measurement  is  to  be  per- 
formed at  1  kHz,  applying  no  more  than  10  V  rms 
to  the  707B. 


8.  Deficiencies  Observed — Action  Taken 

In  a  few  cases,  laboratory  technicians  had  not  at- 
tended Air  Training  Command  school  at  Lowry  Air 
Force  Base  in  Colorado  and  thus  were  not  thoroughly 
familiar  with  the  Electrical  Measurement  Console. 
Certain  areas  of  training  had  not  been  adequately 
covered  so  that  many  unusual  field-measurement  situ- 
ations encountered  could  not  be  correctly  diagnosed. 
With  more  than  half  of  all  reports  received  (80),  the 
deficiencies  can  be  noted  as  follows: 

1.  Temperatures  not  stated  in  degrees  Celsius  as 
requested. 

2.  Improper  ANSI  symbol  usage. 

3.  Error  analyses  not  understood. 

4.  Iron  core  saturation  effects  ignored. 

5.  Incorrect  derivation  of  capacitance  "D" 
factor. 

6.  Effects  of  loading  on  frequency  and  amplitude 
of  complex  signal  generator  not  taken  into  account. 

7.  The  terms  "precision,"  "resolution,"  "accu- 
racy," and  "significant  figure"  not  understood. 

8.  Lack  of  comprehension  of  signal  source 
parameters. 

9.  Problems  created  by  thermal  or  voltaic  emfs, 
ground  loops  (common/normal  mode)  not  recognized. 

10.  Need  for  independent  measurement  schemes 
for  verification  not  appreciated. 

Ignorance  of  the  above  areas  can  damage  test 
equipment  and  reference  standards  and  produce  in- 
accurate or  incomplete  measurements.  Corrective 
action  has  already  been  initiated.  Controlled  Multiple 
Address  Letters  (CMAL's)  are  sent  out  as  required, 
the  curriculum  at  Air  Training  Command  is  being 
modified  to  provide  additional  information  in  the 
above-noted  areas,  and  technicians  from  the  few 
PMEL's  given  limited  certification  are  being  given 
additional  training  at  AGMC  laboratories  by  Metrol- 
ogy Engineering  Division  Engineers. 

9.  Summary  and  Conclusions 

The  implementation  of  a  new  approach  to  metrology 
laboratory  audits  has  reemphasized  the  importance  of 
personnel  in  providing  calibration  support  of  aircraft 
and  weapons  systems.  As  Dr.  J.  L.  Thomas  once  said, 
"You  don't  get  six-dial  measurements  from  a  four-dial 
operator."  The  items  described  are  sufficiently  versa- 
tile so  that  completely  different  problems  can  be  gen- 
erated for  subsequent  tests.  This  can  be  accomplished 
either  by  interconnecting  items  differently  or  by 
actually  changing  readily  available  components. 

A  very  high  degree  of  interest  and  cooperation  was  ob- 
served at  the  audited  laboratories  and  most  were  eager 
to  solve  all  problems,  rather  than  be  limited  to  only  four. 
The  realizable  advantages  and  goals  may  be  stated: 

1.  Improves  morale,  stimulates  original  thinking. 

2.  Provides  in-depth  analysis. 

3.  Assists  in  yearly  Posture  Study. 

4.  Upgrades  capabilities. 

5.  Allows  augmentation  of  available  reference 
material. 

6.  Enables  meaningful  revision  of  training 
programs. 
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The  a-c  Josephson  effect  in  superconductivity  appears  to  be  an  exciting  possibility  for  use  as  a 
quantum  method  of  maintaining  the  volt  as  an  SI  unit,  through  the  relations  2  eV  —  hv.  A  potential- 
divider  resistor  was  devised  for  precise  comparison  of  a  Wesson-cadmium  cell  with  millivolt  outputs 
from  various  solder-drop  junctions  excited  at  36.8  GHz.  The  value  found  for  2  e/h  is  483.5941  ± 
0.0010  MHz/MV69NPL. 
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1.  Introduction 

Part  of  the  work  of  the  Quantum  Metrology  Division 
at  the  NPL  is  devoted  to  measuring  atomic  constants 
and  to  deriving  and  maintaining  the  basic  SI  units. 
Already  the  units  of  length  and  time  are  based  on  quan- 
tum properties,  and  the  ac  Josephson  effect  in  supercon- 
ductivity [1,  2,  3]1  appears  to  be  an  exciting  possibility 
for  use  as  a  quantum  method  of  maintaining  the  volt. 
When  microwaves  of  frequency  v  are  incident  on  the 
barrier  between  two  weakly  linked  superconductors, 
supercurrent  steps  occur  in  the  barrier-current-voltage 
characteristic  which  are  separated  by  a  potential  differ- 
ence V,  given  by  2  eV  =  hv,  where  h/2e  is  the  super- 
conducting magnetic  flux  quantum. 

The  initial  objective  of  our  work  was  the  verification 
of  the  value  obtained  for  2  e/h  from  the  pioneering 
work  [5,  6]  at  the  University  of  Pennsylvania,  to  about 
the  same  level  of  precision.  This  paper  describes  some 
of  the  measurement  techniques  that  have  been  em- 
ployed in  making  the  initial  measurements. 

2.  The  Basic  Measurement  Principle 

As  with  most  quantum  units,  the  voltage  available  foi 
measurement  from  the  Josephson  effect  is  much  smaller 
than  the  maintained  unit,  being  smaller  by  a  factor  of 
between  100  and  2000.  The  basic  measurement  prob- 
lem, therefore,  is  that  of  comparing  two  very  different 
potentials  precisely.  Further,  although  potentials  of  the 
order  of  a  volt  have  been  measurable  to  0.2  fiV  for  some 

1  Figures  in  brackets  indicate  the  literature  references  at  the  end  of 
this  paper. 


time,  there  has  not  been  nearly  as  much  attention  de- 
voted towards  measuring  potentials  at  the  millivolt 
level  to  parts  in  107.  It  is  preferable  that,  in  the  early 
stages  at  least,  as  many  different  methods  as  possible 
are  explored  by  different  groups  of  experimenters: 
agreement  between  two  different  methods  will  add  con- 
siderably to  the  degree  of  confidence  in  either  method. 

Following  the  principles  suggested  by  Dauphinee  [7]. 
of  minimizing  the  number  of  switches  in  the  low-poten- 
tial circuit,  the  comparison  of  the  two  potentials  is 
achieved  simply  by  using  two  resistors  connected  in 
series  as  in  figure  1,  and  balancing  the  Josephson  volt- 
age across  R(,h  and  the  standard-cell  emf  by  the  voltage 
developed  across  Rac-  Unless  the  resistor  ratio,  micro- 
wave frequency,  and  supercurrent  step  number  are  par- 
ticularly chosen,  the  current  L  must  be  adjusted  between 
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Figure  1.  Illustrating  the  principles  of  the 
potentiometer. 
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the  two  balances;  moreover  the  two  currents  must  be 
known  precisely  in  relation  to  one  another.  Considera- 
tions of  the  precision  with  which  the  resistance  ratio 
could  be  measured  led  to  the  conclusion  that  the  resist- 
ance Rac  should  be  about  500  fl.  which,  at  the  time  that 
measurements  commenced,  ruled  out  the  current-com- 
parator potentiometer  [10]  as  the  precisely  variable  cur- 
rent source.  The  basic  methods  of  obtaining  the  vari- 
able current  i-,  were  either  to  maintain  ix  constant  and 
vary  R  >  and  R3,  as  in  the  Julie  I  [8,  9]  inverted  Kelvin- 
Varley-divider  method  used  by  the  Pennsylvania 
group,  [4,  5,  6]  or  to  keep  /,  approximately  constant  and 
vary  7?L.,  deriving  a  feedback  error  signal  to  the  current 
controller  such  that  the  potential  difference  iL./?^  exactly 
equals  a  standard-cell  voltage.  This  error  signal  is 
readily  available  if  the  galvanometer-amplifier  type  of 
current  controller  is  employed.  The  Tinsley-Stabau- 
matic  potentiometer  [11]  which  uses  this  principle  was 
modified  to  suit  our  experiment,  an  important  modifica- 
tion being  the  potential-  divider  resistor,  oabcd,  of  figure 
2.  This  resistor  was  located  outside  the  potentiometer 
and  was  constructed  at  the  NPL. 


9  decade  I  V.D. 
error      2 in  lO'of  part. 
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FIGURE  2.  Circuit  arrangement  used  to  calibrate  the 
divider  resistor. 


3.  Potentiometry 

The  Josephson  junction  is  a  very  broadband  de- 
ice  [2]  the  frequency  response  extending  from  dc  to  the 
infrared,  while  the  nonlinear  behavior  at  the  junction  al- 
lows the  mixing  of  any  frequencies  within  this  range  A 
major  problem  in  any  experiment  involving  small  po- 
tentials is  always  the  elimination  of  potentials  generated 
by  pickup  and  ground  loops.  These  are  particularly 
difficult  to  overcome  when  more  than  one  item  of  equip- 
ment is  connected  to  the  power  lines.  It  was  decided  as 
a  matter  of  principle  to  keep  all  leads  and  apparatus 
using  alternating  current  more  than  3  meters  from  the 
potentiometers  and  cryostat  and  to  use  only  galvanom- 
eters and  galvanometer  amplifiers  for  detecting  balance 
conditions. 


3.1.  The  Voltage  and  Current  Source 

The  voltage  source  in  figure  1,  and  the  source  for  the 
junction  bias  current,  were  both  supplied  by  lead-acid 
batteries.  These  were  charged  overnight  by  electronic 
battery  chargers  to  be  at  the  beginning  of  their  dis- 
charge plateau.  During  the  day  their  emf  was  con- 
stant to  a  few  millivolts. 

3.2.  Current  Controllers  and 
Galvanometer  Amplifiers 

There  are  two  ways  in  which  galvanometer  amplifiers 
and  associated  current  controllers  may  introduce  alter- 
nating currents  when  used  for  sub-ppm  precision 
measurements,  even  though  they  are  screened  from  al- 
ternating magnetic  fields.  The  first  way  arises  if  the 
galvanometer  bulb  is  energized  by  an  alternating  cur- 
rent. The  light  falling  on  the  photocells  is  modulated 
at  the  second-harmonic  of  the  power  frequency,  thus 
introducing  an  alternating  signal  in  the  feedback  loop 
which  then  appears  on  the  current-controller  output. 
This  problem  was  avoided  by  energizing  the  galvanom- 
eter bulbs  with  direct  current.  The  second  way  arises 
from  any  vibration  of  the  galvanometer  spot,  for  such 
vibration  induces  an  alternating  signal  in  the  photocell 
output  which  again  results  in  an  alternating  current  at 
the  output  of  the  current  controller.  (The  effect  of  the 
feedback  loop  is  to  increase  the  frequency  response  of 
the  galvanometer  to  a  few  tens  of  hertz. )  All  galvanom- 
eter amplifiers  were  mounted  on  concrete  blocks  which 
were  in  good  mechanical  contact  with  the  floor.  Fortu- 
nately the  site  of  our  experiment  was  below  ground  level 
and  known  to  be  free  from  vibrations.  The  possibility 
of  resonances  from  acoustic  pickup  effects  was  also 
checked. 

3.3.  The  Potentiometer 

The  essence  of  this  type  of  potentiometer  is  that  a 
standard  cell  is  used  in  the  feedback  loop  of  a  servo- 
system.  Since  the  gain  of  the  system  is  about  2  X  104, 
the  standard  cell  would  supply  current  if  the  system  was 
not  supplying  nearly  the  correct  current  throughout  the 
balancing  operation  (which  R-,  was  switched).  Three 
control  standard  cells  were  employed,  being  switched 
into  operation  in  sequence  as  balance  conditions  were 
approached.  The  final  balancing  cell  was  thereby  pro- 
tected from  any  switching  surges  and  current  drain. 
The  emf  of  this  cell  was  merely  required  to  be  stable 
while  the  Josephson  and  main  standard-cell  voltages 
were  intercompared.  There  was,  in  fact,  a  slow  drift 
throughout  the  day  as  room  temperature  changes  pene- 
trated the  heavy  thermal  lagging  of  the  cell.  This  drift 
was  useful  since  it  meant  that  the  positions  of  the 
switches  varied  from  one  set  of  balances  to  another  and, 
by  making  measurements  on  different  step-numbers,  the 
whole  of  the  potentiometer  entered  into  the  measure- 
ments. 

The  Stabaumatic  potentiometer  and  its  associated 
current  controller  are  designed  to  supply  up  to  20  mA, 
and  a  good  quality  resistor  was  connected  across  od  so 
that  the  combined  resistance  of  it  and  the  divider  re- 
sistor was  about  50  ft.  Possible  nonlinearities  in  the 
current  division  between  the  two  resistors  were  elimi- 
nated by  an  in  situ  ac  calibration  across  the  potential 
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tappings,  using  a  Guildline  current-comparator  poten- 
tiometer[10].  Corrections  to  the  Stabaumatic-potenti- 
ometer  dial  readings  were  calculated  and  applied  as  part 
of  a  computer  program  for  evaluating  the  2e/h  values. 
All  switches  in  critical  parts  of  the  circuit,  including  the 
potentiometer  dial  switches,  were  rotated  20  times  from 
zero  to  full  scale  before  commencing  measurements. 

4.  The  Divider  Resistor 

This  resistor  was  designed  and  constructed  in  the 
NPL  and  was  measured  by  an  ac  measurement  tech- 
nique. 

4.1.  Design 

Measurements  were  made  over  a  range  of  supercur- 
rent  steps  in  the  junction  current-voltage  characteristic 
and  so  the  ratio  of  the  divider  was  chosen  to  be  1,000  :  1. 
Techniques  for  measuring  such  ratios  by  an  ac 
method  [13]  had  recently  become  available  in  the  NPL 
and  it  was  expected  that  an  overall  measurement  pre- 
cision of  better  than  a  part  in  a  million  could  be 
achieved  on  our  ratio  [14].  The  inherent  experimental 
difficulty  of  ensuring  that  the  ac  and  dc  values  of  the 
ratio  were  the  same  had  already  been  solved  by  mem- 
bers of  the  NPL  Electrical  Science  Division  [15].  Our 
divider  was  designed  and  constructed  by  T.  A.  Deacon 
and  M.  John  Swan  respectively.  The  divider  resistor 
was  constructed  from  two  types  of  Evanohm  wire  for 
the  500-  and  0.5-fi  portions  and  thus  behaved  as  two 
separate  resistors  as  far  as  aging  and  temperature  co- 
efficient of  the  divider  ratio  were  concerned.  All  cur- 
rent and  potential  connections  were  welded  to  the  re- 
sistor, and  the  resistors  were  of  bifilar  construction, 
wound  on  a  silicon  glass  laminate  former,  and  mounted 
to  be  at  least  5  cm  from  the  brass  container.  The  termi- 
nations were  brought  as  close  together  as  possible  to 
reduce  temperature  differentials  that  might  have  an  ef- 
fect on  dc. 

Although  most  "dc"  measurements  are  effectively 
made  at  frequencies  between  0.01  and  0.5  Hz,  the  region 
between  these  frequencies  and,  say,  10  Hz  is  that  in 
which  changes  due  to  Peltier  and  other  effects  are  most 
likely  to  occur  if  the  resistor  is  not  sufficiently  well  de- 
signed. The  design  priority  was  given  to  ensuring  that 
the  ac  and  dc  values  were  the  same  and  to  make  the 
ratio  independent  of  ac  frequency,  range  10  to  120  Hz. 
These  objectives  were  satisfied  in  the  completed  divider. 

4.2.  Measurement 

The  equipment  that  was  used  for  measuring  the  di- 
vider ratio  is  illustrated  in  figure  2.  Both  the  divider 
resistor  and  the  9-decade  inductive  ratio  divider  were 
energized  from  the  same  transformer  secondary  wind- 
ing and  the  voltage  across  the  appropriate  divider  ter- 
minals was  balanced  against  that  developed  by  the  in- 
ductive ratio  divider.  The  resistance  R  and  mutual 
inductor  M  were  used  to  inject  a  small  signal  into  the 
detector  to  balance  the  quadrature  component. 

The  inductive  divider  transformer  had  an  error  of 
less  than  2  parts  in  107  of  the  output  at  the  0.001  tap 
over  the  frequency  range  20  to  200  Hz  [14].  The  trans- 
former used  to  isolate  the  detector  was  double-screened ; 
the  secondary  screen  and  the  detector  were  grounded. 


The  primary  screen  was  connected  to  the  guard  trans- 
former divider.  The  guard  voltage  was  set  to  better 
than  a  part  in  a  million,  although  no  effects  were  ap- 
parent when  the  guard  voltage  was  offset  by  parts  in  10'. 

The  ratio  of  the  divider  was  obtained  from  three 
measurements.  In  principle,  of  course,  only  two  meas- 
urements are  required  but  the  grounding  arrangements 
used  for  our  initial  work  made  it  necessary  to  include 
the  resistance  of  the  current  tail  oa  in  the  measurements 
of  the  divider  ratio. 

4.3.  Performance 

The  divider  ratio  was  measured  over  the  frequency 
range  20  to  120  Hz,  and  no  frequency-dependent  varia- 
tions of  the  ratio  were  observed.  ( If  present,  such  ef- 
fects would  generally  vary  linearly  and/or  quadratically 
with  frequency.)  A  change  in  the  voltage  applied  to 
the  divider  from  1  V  to  10  V  did  not  change  the  ratio 
by  as  much  as  2  parts  in  107.  In  making  the  error  as- 
signments to  the  measured  values  of  2e/h  it  was  con- 
sidered that  the  divider  ratio  was  known  to  a  standard 
deviation  of  0.5  ppm  and  that  the  ac  value  represented 
the  dc  value  to  0.5  ppm. 

The  divider  was  put  into  service  shortly  after  con- 
struction and  hence  aged  initially  by  several  ppm  before 
it  settled  to  a  slower  linear  drift  with  time.  Measure- 
ments of  the  divider  ratio  have  been  continued  and  the 
values  obtained  are  within  0.5  ppm  of  the  extrapola- 
tion of  the  curve  fitted  through  the  calibration  points 
pertaining  at  the  time  of  the  2e/h  measurements. 

5.  Measurements 

The  apparatus,  measurements,  and  errors  have  been 
described  in  greater  detail  elsewhere  [16,  17]. 

5.1.  The  Junction 

The  solder-drop  type  of  Josephson  junction  was  used 
for  the  initial  work,  and  so  enabled  the  use  of  a  normal 
liquid-helium  storage  dewar  for  the  cryostat.  This 
simplification  had  a  number  of  advantages:  for  example, 
the  thermal  emf's  in  the  dewar  were  very  stable,  and 
there  were  no  liquid  helium  transfer  problems  since  one 
filling  lasted  for  over  a  week.  Junctions  have  been 
transferred  from  one  storage  dewar  to  another  over  a 
period  of  six  months  and  from  one  site  to  another. 

The  solder-drop  type  of  junction  is  made  by  forming 
a  drop  of  solder  • — •  1  mm  diam,  around  a  length  of 
thinly  oxidized  niobium  wire  —  0.15  mm  diam.  Two 
copper  leads  are  then  embedded  in  the  solder-drop  and 
these,  together  with  the  ends  of  the  niobium  wire,  form 
a  convenient  four-terminal  device.  When  the  room- 
temperature  resistance  of  the  "dry-joint"  formed  be- 
tween the  solder  and  the  niobium  wire  is  about  an  ohm, 
the  yield  of  junctions  showing  Josephson  effects  when 
the  device  is  cooled  to  liquid  helium  temperature  is. 
with  practice,  around  50  percent.  It  is,  incidentally, 
rather  harder  to  produce  a  "dry  joint"  of  about  1-fl 
resistance  than  it  is  to  make  a  well-soldered  joint. 

In  general,  solder-drop  junctions  show  characteris- 
tics of  the  weak-link  type — that  is,  there  is  probably  a 
very  fine  link  of  superconductor  joining  the  niobium 
wire  and  the  solder-drop.  Such  junctions  are  charac- 
terized by  greater  nonlinearities  at  the  junction,  al- 
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though  the  supercurrent  steps  still  appear  at  the  appro- 
priate potentials  V,  governed  by  2eV  =  nhv,  where  h/2e 
is  the  superconducting  magnetic  flux  quantum,  n  is  an 
integer,  and  v  the  applied  microwave  frequency.  As  a 
result  of  the  non-linearities  at  the  barrier  in  the  weak- 
link  type  of  junction,  steps  may  also  appear  at  values  of 
n  corresponding  to  the  ratio  of  two  integers,  for  ex- 
ample, for  n  =  3/2,  5/2,  7/2,  etc.  Such  subharmonic 
steps  are  clearly  visible  in  the  solder-drop  junction 
characteristic  illustrated  in  figure  3.  This  character- 
istic was  obtained  with  an  X— X  pen  recorder. 
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Figure  3.  Example  of  a  solder-drop  current-voltage 
characteristic  when  36.8  GHz  microwaves 
are  incident  on  it. 

Note  the  subharmonic  supercurrent  steps. 


5.2.  Measurements 

The  measurements  of  2e/h  were  made  using  a 
36.8-GHz  phase-locked  klystron  as  the  microwave 
source.  Measurements  were  made  at  lock-points  on 
either  side  of  the  reference-crystal  harmonic,  at  differ- 
ent step  numbers  and  using  different  solder-alloys  for 
the  construction  of  the  junctions.  The  measuring  se- 
quence was  as  follows:  Standard  cell  balance,  junction 
balance,  all  currents  reversed,  followed  by  a  second 
set  of  balances  of  the  junction  and  standard-cell  po- 
tentials. The  temperature-controlled  standard  cells 
were  calibrated  by  J.  J.  Denton  against  the  NPL 
volts  each  day  at  the  beginning  and  end  of  the  2e/h 
measurements. 

5.3.  Thermal  Emfs 

Thermal  emfs  are  an  important  source  of  error  in 
any  dc  measurement  and  are  particularly  so  in  2e/h 
measurements,  where  the  potentials  are  at  the  millivolt 
level.  The  procedure  discussed  above  of  reversing  the 
currents  was  designed  to  eliminate  the  thermal  emfs 
but  it  is,  of  course,  rather  reassuring  if  the  procedure 
can  be  shown  to  eliminate  them.  Fortunately  in  this 
respect,  during  part  of  our  experiment  the  thermal 
emfs  increased  to  around  2  ^.V  instead  of  the  usual 
100  nV  (apparently  resulting  from  a  strain  effect  in 
the  copper  leads  into  the  cryostat).    It  was  found  that 


the  2e//i-values  obtained  with  the  high  thermal  emf 
were  0.8  ±  1.1  ppm  greater  than  the  low  thermal 
values.  The  difference  was  not  statistically  significant 
but  a  standard  deviation  of  1  ppm  was  assigned  to 
the  elimination  of  the  thermal  emfs. 

5.4.  Ground  Loop  Effects 

Ground  loop  effects  are  also  a  problem  in  the  meas- 
urement of  low  dc  potentials.  These  were  avoided  by 
ensuring  that  as  far  as  possible,  all  low  potential  con- 
nections were  copper  to  copper,  and  that  the  con- 
nections were  oil  immersed.  The  effects  of  ac  pickup 
were  reduced  by  operating  all  equipment  within  3  m 
of  the  cryostat  and  potentiometer,  from  dc  sources 
that  were  themselves  well  screened.  After  rearranging 
leads  and  modifying  earthing  arrangements  while  set 
on  a  supercurrent  step,  it  was  concluded  that  ground 
loop  and  pickup  effects  could  not  have  affected  the 
2e/h  values  by  as  much  as  0.5  ppm,  and  this  was 
assigned  as  the  possible  error  from  these  effects. 

6.  Results 

Our  results  and  discussion  of  the  other  sources  of 
error  have  been  discussed  in  detail  in  our  Metrologia 
article  [17]  and  this  discussion  is  not  repeated  here. 
Figure  4  illustrates  the  values  obtained  for  2e/h, 
with  standard  deviations,  for  solder-drop  junctions 
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Figure  4.  The  values  obtained  for  2e/h  with 
solder-drop  junctions  using  different  solder  alloys. 

constructed  from  different  solder-alloys.  Fitting  a 
least  squares  line  through  the  points  indicates  that 
the  values  for  pure  tin  solders  could  be  greater  than 
those  for  pure  lead  solders  by  3  ±  5  ppm.  This  differ- 
ence is,  of  course,  not  statistically  significant. 

The  final  value  for  2e/h  from  these  measurements 

is 

483.5941  ±  0.0010  ( 2.2  in  10* )  MHz//iVG9  npl- 
The  expression  of  our  result  in  terms  of  the  NBS  volt 
cannot  properly  be  made  before  the  results  of  the  1970 
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BIPM  International  Volt  comparisons  are  available. 
On  the  basis  of  the  1967  comparison,  our  values  and 
those  of  the  University  of  Pennsylvania  [6,  18,  19]  are 
unlikely  to  differ  by  more  than  about  0.5  ppm. 

Already,  the  inadequacies  of  the  Weston-cadmium 
cell  as  a  method  of  maintaining  the  volt  are  becoming 
apparent  in  the  Josephson  effect.  The  precisions  of 
the  2e/h  measurements  are  well  beyond  the  precision 
with  which  the  volt  is  known  absolutely  [20] 
(* —  2.7  ppm)  and  measurements  are  improving 
monthly.  It  appears  likely  that  within  a  year  or  so 
sufficient  agreement  between  National  Laboratories 
may  be  achieved  for  the  whole  of  the  experimental 
error  to  be  attributed  to  the  maintenance  of  the  volt 
by  the  Weston-cadmium  cell.  At  this  stage  it  will  be 
possible  to  maintain  the  volt  by  the  Josephson  effect. 
Beyond  that,  there  lies  the  possibility  that  the  Joseph- 
son  effect  might  ultimately  lead  to  a  redefinition  of 
one  of  the  SI  units. 


This  work  forms  part  of  the  research  program  of 
the  National  Physical  Laboratory. 
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Advances  in  electronic  technology  have  made  possible  significant  improvements  in  both  accuracy 
and  precision  of  measurements.  Some  comments  will  be  made  on  the  increase  in  accuracy  through 
automatic  application  of  data  corrections  (which  were  previously  too  difficult  to  be  practicable)  and 
the  elimination  of  many  "human"  errors.  Better  precision  coming  from  increased  speed  of  measure- 
ment and  improved  control  of  the  environment  of  the  measurement  will  be  discussed. 

Key  words:  Attenuation  measurement;  automatic  data  correction;  auxiliary  receiver  for  microwave 
measurements;  dual  admittance  Woods  bridge;  Engen  microcalorimeter;  rf  coaxial  impedance 
standards. 


1.  Introduction 

Advances  in  electronic  technology  have  made  pos- 
sible substantial  improvement  of  both  accuracy  and 
precision  through  better  control  of  the  total  environ- 
ment of  a  measurement.  In  particular,  reduction  of 
human  involvement  and  automatic  application  of  cor- 
rections to  data  have  increased  accuracy,  while  shorter 
measurement  times  and  automation  of  manually  oper- 
ated instruments  have  given  us  greater  precision. 

2.  "Human"  Errors 

The  experimenter's  human  characteristics  have  tra- 
ditionally set  limits  on  precision  and  accuracy  of  a 
measurement.  In  addition  to  basic  defects  in  precision 
sensing,  memory,  physical  stability,  speed,  and  dura- 
bility, his  greatest  asset — intelligence — makes  him  un- 
able to  perform  reliably  and  consistently  a  simple  task 
without  frequent  error.  Now,  there  has  begun  a  tre- 
mendous upsurge  in  the  use  of  computers  to  replace 
man  in  the  rapid  performance  of  simple  tasks. 

The  man  in  charge  of  a  calibration  can  now  have  an 
unintelligent  machine  of  incredible  speed  and  great  re- 
liability to  replace  an  intelligent  technician.  Doubtless, 
in  the  early  stages  of  such  a  development,  great  effort  is 
necessary  on  the  part  of  the  supervisor  if  he  is  to  com- 
municate with  this  unintelligent  beast  in  its  strange 
language.  However,  such  extra  effort  will  pay  for  itself 
in  the  insight  gained  into  the  calibration  procedure. 
Perhaps  a  better  and  more  basic  measurement  will 
result. 

The  whole  aspect  of  standard  calibration  procedures 
can  be  improved  by  the  sharing  of  computer  programs 
by  different  laboratories.  As  programs  are  modified 
and  become  accepted  by  the  different  laboratories, 
many  of  our  calibrations  will  be  performed  by  a  truly 
standardized  procedure. 


3.  Data  Corrections 

One  of  the  principal  uses  of  computer  involvement 
in  a  calibration  laboratory  is  for  the  automatic  applica- 
tion of  data  corrections.  In  January  of  1950,  D.  Woods 
[l]1  gave  a  complete  description  of  a  dual-admittance 
bridge  to  be  used  for  measurement  of  shunt  capacitance 
from  — 50  to  +50  pF  and  shunt  conductance  of  50 
mmho  or  less  in  the  frequency  range  3  to  300  MHz. 
The  excellent  0.1  percent  accuracy  of  conductance 
measurement  and  0.2  percent  accuracy  of  complex  ad- 
mittance were  made  possible  by  painstaking  care  in  as- 
sessment of  all  sources  of  error  in  the  bridge.  Many  of 
the  errors  were  made  negligible  by  careful  mechanical 
and  electrical  design  and  the  use  of  external  calibration 
standards.  The  considerable  number  of  errors  remain- 
ing were  corrected  for  by  the  use  of  a  prepared  calcula- 
tion table  which  guided  the  worker  through  the  steps 
for  the  corrections.  With  the  help  of  this  table  and  an 
electric  calculating  machine,  the  results  for  a  set  of  six 
measurements  at  one  frequency  could  be  obtained  in 
two  hours  time. 

This  illustrates  what  was  once  a  basic  philosophy  of 
a  standards  laboratory — time  is  not  a  major  considera- 
tion. If  you  imagine  the  addition  of  automatic  readout 
and  computer  application  of  corrections  for  the  Woods 
bridge,  you  could  further  improve  accuracy  by  perform- 
ing more  measurements  using  external  standards  with 
no  need  to  improve  the  bridge  itself.  As  is  the  case 
with  the  automatic  measurement  system  described  by 
Adam  [2],  the  ultimate  limitations  on  accuracy  would 
be  set  by  the  repeatability  of  the  bridge  and  the  char- 
acteristics of  the  external  standards  and  their  connec- 
tion to  the  bridge. 

There  is  another  factor  to  be  considered  in  respect 


1  Figures  in  brackets  indicate  the  literature  references  at  the  end  of 
this  paper. 
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to  the  use  of  automatic  data  corrections.  A  measure- 
ment technique  which  would  be  quite  impractical  and 
tedious  for  manual  operation  might  be  chosen  because 
of  its  mechanical  simplicity  or  the  criterion  of  greater 
accuracy.  An  example  of  this  is  the  technique  for  co- 
axial impedance  measurement  described  by  Jurkus  [3]. 
This  system  has  been  used  to  calibrate  coaxial  termina- 
tions in  14-mm  precision  coaxial  transmission  line  at 
about  100  frequencies  per  octave  over  the  frequency 
range  1  to  8  GHz.  It  is  a  semiautomatic  system  using  a 
fixed  probe  in  the  transmission  line  and  makes  full  use 
of  the  computer  for  analysis  of  the  data  including  cor- 
rections for  source  mismatch  and  loss  in  the  reference 
air  lines  used  as  impedance  standards. 

4.  Improved  Precision 

The  time  required  to  perform  a  measurement  auto- 
matically is  usually  dramatically  less  than  required  for 
a  manual  one.  Thus,  statistical  control  of  the  measure- 
ment becomes  a  distinct  possibility  because  large  num- 
bers of  measurements  may  be  performed  within  a  rea- 
sonable time.  Also,  precision  should  improve,  since 
the  magnitude  of  certain  parameters,  especially  those 
affected  by  temperature,  will  be  more  constant  over  a 
short  period  of  time  than  over  a  long  period. 

The  instrument  makers,  realizing  that  the  human 
being  is  becoming  less  involved  in  the  measurement 
process,  are  rapidly  converting  their  instruments.  The 
attention  to  and  compensation  for  the  defects  of  man 
are  being  replaced  by  a  similar  concern  for  the  vagaries 
of  the  computer.  Man,  however,  can  benefit  from  some 
items  such  as  programmable  sources,  meters  with  digital 
readout,  or  meters  with  an  automatic  zero  adjustment. 
We  have  benefitted  in  our  laboratory  from  an  automatic 
zero  adjustment  on  a  thermistor  bridge  used  to  operate 
a  thermistor  mount  for  rf  power  measurement.  This 
benefit  arose  in  connection  with  the  insertion  loss  meas- 
urement of  a  20-dB  fixed  waveguide  attenuator  of  the 
multihole  directional  coupler  type  and  used  as  an  inter- 
laboratory  transfer  standard  of  attenuation  at  a  fre- 
quency of  10  GHz. 

As  is  sometimes  the  case  in  a  standards  laboratory, 
the  stable  microwave  source  and  the  precision  micro- 
wave receiver  which  form  part  of  the  environment  for 
our  best  attenuation  measurements  achieve,  in  part, 
their  stability  from  their  massive  size.  This  size  can 
make  the  insertion  and  removal  of  the  attenuator  dif- 


ficult and  time-consuming  because  of  the  severe  require- 
ments on  alinement  and  positioning  of  the  connecting 
flanges.  Larson  [4]  has  suggested  modification  of  the 
attenuator  so  that  the  input  and  output  parts  are  in  line 
and  have  the  same  axis.  This  will  reduce  the  required 
relative  motion  of  the  source  and  receiver,  especially 
when  a  piece  of  waveguide  having  the  same  length  as 
the  attenuator  is  used  to  replace  the  attenuator  and  a 
differential  insertion  loss  measurement  is  performed. 

It  is  possible,  however,  to  use  a  small  auxiliary  re- 
ceiver as  shown  in  figure  1.  The  auxiliary  receiver  is  a 
thermistor  mount  and  a  tuner,  and  is  light  enough  to  be 
supported  by  the  flange  to  which  it  is  connected.  The 
change  in  output  power  from  the  rf  source  required  to 
maintain  a  level  of  0.4  mW  into  the  thermistor  mount 
with  the  attenuator  inserted  or  removed  was  measured 
by  the  precision  receiver.  The  rf  source  output  was 
reduced  to  zero  and  the  power  meter  automatically 
zeroed  by  depressing  a  front  panel  switch  immediately 
prior  to  each  measurement.  The  standard  deviation  of 
the  mean  of  10  measurements  on  a  20-dB  attenuator 
was  0.0003  dB,  which  is  about  an  order  better  than  we 
could  obtain  with  an  instrument  having  a  manual  zero, 
used  in  our  air-conditioned  laboratory  at  the  level  of 
0.4  mW. 

5.  Role  of  the  Standards  Laboratory 

Ultimately  a  computer-assisted  measurement  system 
will  outperform  a  manual  system  to  such  an  extent  that 
the  standards  laboratories  will  be  forced  to  participate 
fully  in  the  revolution.  For  the  present,  the  standards 
laboratory  will  be  faced  with  the  need  for  calibration 
of  the  standards  which  are  used  as  references  in  the 
automatic  system.  The  calibration  accuracy  probably 
need  not  be  improved,  but  the  range  of  the  calibration 
will  need  to  be  increased  as  reliance  placed  on  the  in- 
strument is  shifted  to  the  standards.  I  would  like  to 
end  this  paper  with  a  few  words  about  two  particular  rf 
standards  which  require  calibration  with  the  presently 
available  accuracy  but  at  many  more  frequencies  so 
that  the  parameters  of  the  standard  are  well  known 
throughout  its  frequency  range. 

An  ideal  working  or  transfer  standard  for  rf  coaxial 
impedance  measurement  is  the  coaxial  resistive  termi- 
nation. It  is  simple,  rugged,  stable,  and  covers  a  large 
frequency  range.  This  is  in  contrast  with  quarter- 
wavelength  coaxial  air  lines,  which  are  very  narrow 
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Figure  1.  Measurement  of  insertion  loss  with  a  thermistor  mount  and 
power  meter  combination  used  as  an  auxiliary  receiver. 
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band,  and  coaxial  sliding  loads  in  precision  air  lines, 
which  tend  to  be  expensive  and  somewhat  limited  in 
frequency  coverage.  The  calibration  technique  of 
Jurkus  [3],  mentioned  above,  provides  a  sufficient  num- 
ber of  calibration  frequencies  so  that  the  calibrated  co- 
axial termination  may  be  used  at  any  frequency  from  1 
to  8  GHz  with  the  aid  of  linear  interpolation  of  the  data. 
For  the  "non-automated"  who  still  measure  impedance 
at  one  frequency  with  a  slotted  line  having  mechanical 
drive  and  X-Y  recorder  for  readout,  these  calibrated 
terminations  can  provide  a  quick,  accurate  check  of 
residual  VSWR  of  the  slotted  line,  again  at  the  fre- 
quency of  measurement. 

The  second  rf  standard  which  needs  calibration  at  a 
great  number  of  frequencies  is  the  thermistor  mount 
which  is  used  as  a  transfer  standard  of  power.  The 
ideal  calibration  environment  for  accurate  measure- 
ment, at  several  frequencies  of  the  effective  efficiency 
■q  of  a  thermistor  mount,  is  the  Engen  microcalorimeter 
[5],  since  this  calorimeter  has  a  high-order  sensitivity 
to  (1  —  rj)  and  a  low-order  sensitivity  to  mismatch 
error.  We  have  used  a  calorimeter  based  on  the  Engen 
design  to  measure  the  effective  efficiency  of  X-band  and 
Ku-Band  thermistor  mounts  at  a  sufficient  number  of 
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Figure  2.  Effective  efficiency  (77)  of  (a)  an  X-band 
waveguide  thermistor  mount;  {b)  a  Ku- band  waveguide 
thermistor  mount. 


calibration  frequencies  to  permit  linear  interpolation 
(see  fig.  2).  The  calibration  time  was  lengthy,  how- 
ever, and  we  are  currently  constructing  an  X-band 
calorimeter  of  modified  design  with  a  view  towards  re- 
duction of  the  time  involved. 
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DESIGN  AND  DEVELOPMENT  THROUGH  METROLOGY 
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The  National  Physical  Laboratory  of  India,  in  implementing  the  1956  Standards  of  Weights  and 
Measures  Act,  adjusted  and  calibrated  mass  standards  of  50  to  10"  kg  for  250  State  Laboratories. 
Most  of  the  5000  balances  needed  by  Inspectors  were  designed  and  fabricated  by  four  commercial 
manufacturers  in  India,  with  specifications,  technician  training,  and  performance  testing  supplied  by 
NPL.  With  this  experience,  two  manufacturers  designed  and  fabricated  the  higher  precision 
balances  for  District-level  laboratories.  The  technique  is  recommended  to  developing  countries  faced 
with  similar  projects. 

Key  words:  Designing  to  specification;  fabrication  of  balances;  legal  metrology;  mass  standards; 
NPL  of  India. 


1.  Legal  Metrology  in  India 

The  Standards  of  Weights  and  Measures  Act  was 
passed  in  1956  and  thus,  after  many  centuries,  India 
once  again  had  a  uniform  system  of  measurement 
throughout  the  country.  The  establishment  of  the 
standards  of  weights  and  measures  was  made  the  re- 
sponsibility of  the  Central  Government,  whereas  the 
enforcement  of  various  provisions  of  the  Law  was  made 
the  responsibility  of  all  the  State  governments. 

To  organize  the  enforcement  machinery  simultane- 
ously in  about  25  States  and  Union  Territories  was  no 
simple  matter,  particularly  because  industry  in  the 
country  was  still  in  a  developing  stage.  With  proper 
planning  and  phasing  of  an  implementation  program,  it 
became  possible  to  procure  most  of  the  required  equip- 
ment from  local  sources  only. 

At  about  this  time,  through  a  happy  coincidence,  the 
Weights  and  Measures  Division  of  the  National  Physical 
Laboratory  of  India  had  been  expanded  in  a  big  way  by 
equipping  it  with  a  number  of  important  measuring  in- 
struments and  other  important  items  of  equipment. 
Most  of  these  were  obtained  through  the  courtesy  of  the 
Technical  Cooperation  Mission  of  the  United  States  of 
America  under  Agreement  No.  26.  The  Laboratory 
was,  therefore,  in  a  position  to  take  up  all  technical 
work  relating  to  the  implementation  of  the  1956  Act  and 
was,  in  fact,  associated  from  the  beginning  with  the 
Central  Ministry  concerned  and  all  the  State  depart- 
ments. 

The  NPL's  first  assignment  was  to  help  in  drafting 
the  1956  Act.  The  next  important  assignment  was  to 
work  out  the  shapes  and  dimensions  of  various  grades 
of  standards  (reference,  secondary,  and  working),  as 
well  as  commercial  grades  of  new  weights  (requiring 
some  four  grades),  length  measures,  and  volume  meas- 
ures. One  of  the  chief  requirements  for  all  the  new 
weights  and  measures  was  that  these  should  possess 


distinctly  different  shapes  from  all  those  already  in  use 
in  the  country.  The  third  important  assignment  for  the 
Laboratory  was  to  recommend  suitable  limits  of  permis- 
sible errors  for  all  categories  of  weights  and  measures, 
from  reference  standard  grade  down  to  the  coarsest 
commercial  grade.  The  fourth  assignment  was  to  pre- 
pare about  two  dozen  demonstration  sets  of  metric 
weights  and  measures,  in  accordance  with  the  new 
shapes  and  dimensions,  for  exhibition  in  different  States 
and  Union  Territories.  This  was  done  with  a  view  to 
familiarizing  the  general  public  and  the  manufacturers 
with  the  new  standards.  Also  manufacturers  were 
given  all  necessary  help  to  enable  them  to  go  ahead 
with  the  manufacture  of  the  standards.  All  this  work 
took  a  couple  of  years  after  the  enactment  of  the  new 
law. 

Another  important  assignment  for  the  Laboratory 
was  to  provide  necessary  training  in  high-precision  tests 
and  measurements  to  senior  officers  and  to  the  Inspec- 
tors of  Weights  and  Measures  responsible  for  enforce- 
ment work  in  the  States.  This  work  is  of  a  continuing 
nature  and  has  been  going  on  intermittently  all  these 
years.  About  600  officers  have  already  received  this 
training. 

2.  Balances  for  the  Enforcement 
Laboratories 

The  question  of  equipping  the  State  Enforcement 
Laboratories  was  looked  into  along  with  other  jobs.  It 
was  estimated  that  about  250  laboratories  with  about 
1000  Inspectors  of  Weights  and  Measures  would  be  re- 
quired for  effective  enforcement  of  the  Act.  These 
laboratories  were  to  be  provided  immediately  with  all 
necessary  standards,  measuring  equipment,  and  bal- 
ances for  the  verification  of  commercial-grade  weights 
from  50  kg  down  to  a  milligram.  It  was  decided  that 
the  weights  would  be  manufactured  by  the  India  Gov- 
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ernment  Mint.  The  assignment  given  to  the  Laboratory 
in  connection  with  these  standards  was  to  adjust  and 
calibrate  them  and  to  issue  certificates  for  each  item 
individually  before  they  were  supplied  to  the  State 
departments. 

Then  came  the  question  of  providing  some  5000  bal- 
ances to  these  State  laboratories.  The  possibility  of 
importing  these  was  ruled  out,  as  this  required  a  foreign 
exchange  of  about  25  million  rupees.  To  make  a  quick 
start,  however,  it  was  considered  necessary  to  import 
about  1000  balances.  The  purchase  of  the  remaining 
balances  was  deferred  with  a  view  to  trying  to  get  these 
manufactured  in  our  country.  A  committee,  with  the 
author  of  this  paper  as  the  convener,  was  set  up  in  1958 
to  look  into  the  possibilities  of  getting  the  required  type 
of  balances  made  in  India. 

Taking  all  the  relevant  factors  into  consideration,  the 
committee  decided  that  a  set  of  balances  should  consist 
of  50-kg,  5-kg,  200-g.  and  2-g  capacity  balances. 
Broad  specifications  drawn  up  for  these  balances  are 
given  in  table  1.  Two  types  of  balances  were  required: 
(i)  Indoor-type  balances,  to  be  permanently  set  up 
in  the  laboratories.  All  four  balances  in  the  set  were 
required  to  be  provided  with  the  usual  type  of  glass  case. 


Table  1 


Approx. 

Work  Load  on 

Sensitivity 

Beam 

each  Balance 

Capacity 

Reciprocal 

Length 

(Nos.  of  Denom. ) 

mg/ div. 

50  kg 

100 

About  750  mm 

3 

5  kg 

10 

250-300  mm 

4 

200  g 

1 

150-200  mm 

6 

2  g 

0.02 

120-150  mm 

11 

( ii  I  Portable  balances,  for  outdoor  use.  These 
should  be  easily  dismantled  and  secured  in  their  carry- 
ing cases.  They  should  be  sufficiently  robust  to  remain 
undamaged  even  when  the  packages  were  transported 
over  rough  roads.  They  should  be  easily  set  up  at  camp 
laboratories.  The  50-kg  balance  was  not  to  be  pro- 
vided with  the  usual  type  of  supporting  pillar  but  with 
a  sufficiently  rigid  foldable  tripod  stand  for  suspending 
the  balance.  No  glass  case  was  required  for  this  bal- 
ance. 

After  laying  down  these  specifications,  the  project 
was  entrusted  to  the  author,  to  be  tackled  by  the  Labora- 
tory in  the  manner  considered  best.  Advantage  was 
taken  of  earlier  experience  regarding  Indian  balances. 
The  laboratory  had  had  occasion  to  examine  a  few  lo- 
cally manufactured  chemical  balances  some  years  ear- 
lier. (Small-scale  production  of  such  balances  had  been 
going  on  in  the  country  from  the  beginning  of  this 
century.)  On  testing  these  balances  it  was  found  that, 
though  these  were  all  right  for  college  purposes,  they 
lacked  many  characteristics  needed  for  precision  work. 
Letters  were  then  sent  to  the  leading  manufacturers 
telling  them  that  the  Laboratory  would  be  able  to  sug- 
gest improvements  in  the  quality  of  their  balances  pro- 
vided they  sent  them  to  us  for  close  and  critical  exami- 
nation. Those  manufacturers  who  responded  to  our 
offer  did  benefit  by  our  advice,  and  their  balances  were 


subsequently  considered  superior  to  those  of  others. 
Our  general  finding  was  that,  although  the  manufactur- 
ers had  some  of  the  finest  craftsmen  on  their  staffs, 
proper  technical  guidance  was  sometimes  lacking. 

The  representatives  of  these  manufacturers  were 
trained  in  making  good-quality  knife  edges  and  bearing 
planes  for  precision  balances,  as  well  as  in  carrying  out 
various  performance  tests  on  these.  They  were  asked 
to  give  complete  tests  to  the  balances  before  sending 
them  to  us.  Based  on  our  earlier  experience  with  local 
balances  we  asked  them  to  take  special  care  with  re- 
spect to 

1.  choice  of  proper  material  for  the  construction  of 
knife  edges  and  bearing  planes;  proper  hardening  of 
the  knife  edges  where  steel  was  used. 

2.  accurate  lapping,  correct  positioning,  and  rigid 
mounting  of  the  planes  and  knife-edges. 

3.  adequate  rigidity  of  the  beam,  so  that  there  was 
not  much  variation  with  load  in  the  sensitivity  of  the 
balance. 

The  procedure  planned  was  that  the  manufacturers 
were  to  fabricate  sample  balances  to  the  best  of  their 
ability  and  submit  the  same  to  the  Laboratory  for  criti- 
cal examination.  This  would  enable  us  to  locate  any 
defects.  Necessary  advice  would  then  be  given  for  the 
elimination  of  these  defects  in  their  subsequent  attempts. 
This  process  would  continue  until  their  balances  came 
up  to  the  required  standard. 

Within  a  few  months  the  manufacturers  were  able  to 
work  out  details  of  their  designs,  fabricate  the  balances, 
and  submit  their  first  models  for  test.  Detailed  examina- 
tion of  these  balances  was  carried  out  with  a  view  to 
discovering  any  design  or  constructional  defects,  such 
as  in  overall  workmanship,  rigidity  of  the  fixing  ar- 
rangements for  the  planes  and  knife  edges,  and  the 
proper  positioning  of  these.  The  following  tests  were 
carried  out: 

1.  Determination  of  sensitivity  and  period  of  swing 
under  (i)  no  load,  (ii)  half  load,  and  (in)  full  load 
conditions. 

2.  Consistency  of  performance,  i.e.,  the  effect  of  re- 
leasing, arresting,  and  re-releasing  the  beam  a  number 
of  times,  and  noting  down  the  rest  point.  Some  20 
observations  were  made  in  succession. 

3.  Accuracy  of  weighing.  Rigidity  of  the  beam, 
proper  hardness,  and  accuracy  of  the  lapping  of  the 
knife  edges,  etc..  were  inferred  from  the  results  of  test. 

4.  Determination  of  the  relative  lengths  of  the  arms. 

5.  Stability  of  rest  point  over  a  long  period  of  time 
( at  least  two  to  three  weeks)  and  under  varying  con- 
ditions of  temperature. 

6.  Uniformity  of  the  rider  scale,  if  provided. 

The  results  of  this  experiment  were  very  encouraging, 
as  some  manufacturers  were  successful  in  the  very  first 
attempt  to  produce  nearly  the  required  quality  of  in- 
strument. Representatives  of  the  manufacturers  were  ad- 
vised to  be  present  during  tests.  They  were  given  neces- 
sary instructions  for  further  improving  the  performance 
of  their  balances.  They  then  produced  their  second 
models,  which  were  an  improvement  over  the  first  ones. 
The  whole  process  of  testing  and  giving  instructions  to 
the  manufacturers  was  repeated,  and  they  were  asked  to 
prepare  yet  another  model.    This  continued  until  they 
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were  able  to  produce  the  desired  quality.  Out  of  the  six 
manufacturers  who  participated  in  the  project,  two 
gave  up  after  making  two  or  three  unsuccessful  at- 
tempts. Of  the  remaining  four,  three  succeeded  in 
producing  the  desired  quality  after  making  three  or 
four  attempts.  This  was  within  a  year  and  a  half  of 
starting  the  project.  The  fourth  manufacturer  took 
somewhat  longer,  but  he  too  qualified  after  making  a 
few  more  attempts. 

These  four  manufacturers  met  the  entire  demand  an- 
ticipated for  balances  for  the  State  enforcement  labora- 
tories within  the  next  three  years  or  so. 

3.  Higher  Precision  Balances 

As  the  enforcement  laboratories  became  more  ad- 
vanced in  their  work,  they  required  more  sophisticated 
items  of  equipment.  They  now  required  balances  with 
higher  accuracy  and  sensitivity  which  would  be  suitable 
for  the  periodical  verification  of  working  standard 
weights  against  the  secondary  standards.  These  bal- 
ances were  to  be  installed  at  the  District-level  labora- 
tories. Taking  into  account  the  permissible  errors  on 
working  standard  weights,  it  was  decided  to  have  five 
balances  in  the  set,  with  broad  specifications  as  shown 
in  table  2. 

These  specifications  were  sent  to  the  four  manufac- 
turers who  had  been  successful  in  producing  balances 
for  the  use  of  Inspectors.  They  were  asked  if  they  were 
willing  to  follow  the  same  old  procedure  for  producing 
the  more  precise  type  of  balance.  Two  manufacturers 
who  possessed  the  experience  of  manufacturing  a  larger 
of  number  of  balances  than  the  other  two,  and  had  con- 
sequently gained  more  confidence,  agreed  to  take  up  the 
development  of  these  balances. 

The  same  procedure  was  followed,  and  within  one 
year  these  balances  also  were  successfully  produced  by 
both  manufacturers.  Production  is  in  full  swing  now, 
and  some  60  sets  have  already  been  produced  and  sup- 
plied to'  different  States. 


Table  2 


Approx. 

Approx. 

Minimum 

Approx. 

Width 

Clearance 

Sensitivity 

Scale 

Beam 

Across 

Above 

Capacity  Reciprocal 

Division 

Length 

Pans 

Pans 

mg/ div. 

mm 

mm 

mm 

mm 

20  kg 

25 

2 

750 

300 

700 

5  kg 

7.5 

1.5 

350 

175 

325 

1kg 

1.5 

1.0 

250 

135 

225 

50  g 

0.4 

1.0 

125 

60 

100 

2  g 

0.02 

0.75 

110 

50 
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Within  the  next  year  or  so,  work  in  connection  with 
balances  of  still  higher  precision  will  be  undertaken. 
These  balances,  which  will  be  installed  at  the  State 
Headquarters  Laboratories  only,  will  be  employed  for 
the  periodical  verification  of  the  secondary  standard 
weights  against  the  reference  standards.  Specifications 
for  these  balances  will  be  considerably  more  rigid  than 
those  for  the  two  types  already  produced.  These  bal- 
ances will  be  required  to  be  fitted  with  an  optical  lever 


system  for  increasing  their  sensitivity.  The  accuracy 
requirements,  as  well  as  the  minimum  requirements  for 
the  equality  of  arms,  will  also  be  prescribed.  Making 
of  these  balances  will  be  a  somewhat  tougher  job.  How- 
ever, with  the  confidence  the  two  manufacturers  have 
now  gained,  they  should  be  able  to  make  these. 

4.  About  the  Method 

The  method  which  a  research  laboratory  would  have 
adopted  in  such  a  situation  would  be  to  put  a  large 
number  of  designers  on  the  job,  get  the  trial  models 
made,  and  put  these  to  practical  tests.  Depending  on 
the  results  of  tests,  the  designs  might  require  further 
modification.  After  incorporating  all  the  desired  re- 
quirements in  the  finalized  design,  a  prototype  would 
be  constructed  to  hand  over  to  the  industry.  It  was 
realized  that,  with  the  resources  at  our  disposal,  or  for 
that  matter  at  the  disposal  of  any  medium-sized  labora- 
tory, and  to  work  simultaneously  in  many  varied  fields, 
it  might  not  be  possible  to  put  a  large  team  of  designers 
and  mechanics  on  any  single  project.  It  was  estimated 
that  if  we  adopted  this  method  it  would  require  at  least 
five  years  to  produce  any  tangible  results. 

Circumstances  compelled  us  to  think  of  an  inexpen- 
sive method  which  would  give  the  desired  results  within 
a  short  time,  preferably  a  year  or  so,  because  the  imple- 
mentation program  could  not  wait  five  years.  The 
method  followed  by  us  as  described  above  has  certainly 
been  efficacious.  Essentially,  the  detailed  design  work 
is  done  by  the  manufacturers,  with  necessary  guidance 
provided  to  them  from  time  to  time  on  the  basis  of  tests 
and  precise  measurements  on  their  product  by  the  col- 
laborating research  institute.  This  method  has  proved 
its  merits  on  two  occasions  already.  It  resulted  in  the 
saving  of  considerable  foreign  exchange  and  in  estab- 
lishing a  flourishing  balance  industry  in  the  country, 
and  it  has  proved  to  be  a  promising  technique  for  han- 
dling design  projects.  Together,  the  research  institute 
and  the  industry  have  thus  been  able  to  achieve  easily 
what  each  individually  might  have  found  quite  difficult 
to  tackle. 

It  is  felt  that  this  method,  although  employed  out  of 
necessity,  can  be  used  by  developing  countries  with  con- 
siderable benefit,  by  adapting  it  for  any  similar  project. 
The  advantages  offered  by  this  method  are: 

( 1 )  It  saves  considerable  time  in  the  successful  com- 
pletion of  a  project  for  the  following  reasons: 

f  a )  By  use  of  the  facilities  of  a  number  of  manufac- 
turers, a  number  of  designers  and  precision  mechanics 
much  greater  than  even  a  big  laboratory  could  afford 
to  employ  can  be  associated  with  the  project. 

(b)  Involving  a  number  of  manufacturers  in  the 
same  project  creates  a  spirit  of  competition,  and  each 
manufacturer  tries  to  achieve  results  more  quickly  than 
the  others. 

(2)  It  assures  a  good  design  for  the  product  for  the 
following  reasons: 

(a)  The  manufacturers'  designers  as  well  as  me- 
chanics who  participate  in  the  project  can  do  the  job 
better,  because  of  association  with  the  trade  concerned, 
than  can  the  general  run  of  designers  and  mechanics. 

( b )  Since  the  design  was  not  frozen  by  the  re- 
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search  institute,  each  manufacturer  uses  his  ingenuity 
to  make  his  product  better  than  that  of  his  rivals. 
3.  It  reduces  cost  very  considerably : 

(a)  Instead  of  spending  on  the  project,  the  research 
institute  may  even  earn  test  fees  by  testing  an  item  each 


time  it  is  received  from  the  manufacturers. 

(b)  Manufacturers  will  plan  their  design  and  fab- 
rication work  in  the  most  economical  manner  by  suit- 
ably fitting  this  work  into  the  rest  of  their  program. 
In  this  way  they  incur  little  extra  expenditure. 


Figures  1  and  2.  Two  views  of  the  NPL  balance-testing  section. 
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Secretary  General,  Verband  Deutscher  Elektrotechniker,  Frankfurt,  W.  Germany 

Automatic  recorders  with  uncertainties  no  greater  than  0.1  dB  up  to  18  MHz  are  used  in  one 
channel  of  a  microwave  link  to  measure  effective  equivalence  and  noise  level  to  provide  production 
calibration  control  for  filters  and  attenuators.    Details  of  two  German-made  systems  are  described. 

Key  words:  Automatic  noise-level  recorders;  calibration  of  carrier-frequency  filters;  microwave 
transmission  lines;  Siemens-Halske  AEG;  Wandel  &  Goltermann. 


The  progress  of  developing  techniques  very  often 
depends  on  communications.  All  prognoses  for  tele- 
communication expenditures,  especially  those  of  the 
Bell  System,  show  the  necessity  for  a  very  great  num- 
ber of  links  in  the  national,  as  well  as  in  the  inter- 
national, field. 

Beside  the  quantity  of  links,  their  quality  must  also 
be  considered.  It  is  known  that  each  circuit  able  to  be 
linked  with  other  circuits  for  public  use  should  hold 
special  requirements  elaborated  by  the  International 
Consultative  Committees  (CCI)  of  the  International 
Telecommunication  Union.  It  may  be  that  in  the  United 
States  of  America  it  will  not  be  necessary  to  follow  these 
requirements  because  only  a  small  number  of  links  will 
be  connected  with  international  lines.  In  Germany, 
however,  we  are  in  a  quite  different  position.  Our 
country  is  very  small.  Practically  100  percent  of  our 
traffic  is  automatically  operated  by  the  user.  This  is 
true  also  for  international,  and  recently  was  inaugurated 
for  intercontinental,  service.  We  are  surrounded  by 
many  other  small  countries.  All  the  very  heavy  tele- 
communication traffic  between  these  countries  is 
switched  automatically  between  countries  and  is  nor- 
mally considered  as  foreign  exchange.  Therefore,  the 
quality  of  the  circuits  has  to  fulfill  the  CCI  require- 
ments. The  measurements  should  be  valid  for  micro- 
wave links  as  well  as  for  carrier-frequency  cables. 

The  very  rapid  increase  in  the  number  of  lines  sub- 
ject to  the  necessary  high  quality  controls  is  the  reason 
for  developing  remotely  controlled  automatic  measure- 
ment equipment  for  production  and  acceptance  tests,  as 
well  as  for  maintenance.  It  would  be  impossible  to  get 
a  sufficient  staff  to  run  the  systems  on  any  other  basis. 

The  ever-increasing  demands  for  quality  of  telecom- 
munication-transmission systems  require  the  use  of 
high-precision  measuring  equipment.  In  addition  to 
this,  the  measurements  must  be  made  in  the  most  simple, 
exact,  and  time-saving  manner,  preferably  auto- 
matically. 


It  seems  unnecessary  to  mention  the  automatic  noise- 
level  recorder  for  checking  the  statistical  values  of  noise 
in  specific  circuits.  In  the  cable  or  microwave  link  of  a 
multichannel  carrier-frequency  system,  one  channel  is 
used  as  a  measurement  or  control  channel.  Then  it  is 
possible  to  count  and  to  print  the  median  noise-level 
values  for  one  hour,  for  one  minute,  or  for  5  millisec- 
onds. The  measurement  limits  are  between  100  and 
100,000  pW.  This  equipment  works  together  with  a 
small  computer  to  mark  the  values  of  out-of-tolerance 
limits  and  to  indicate  if  the  overall  system  is  still  usable. 

On  production  and  acceptance  tests,  the  carrier- 
frequency  filters  take  most  of  the  testing  time  because 
each  filter  must  be  checked  at  many  frequencies.  The 
filters  these  days  are  mass  produced  with  very  high  ac- 
curacy, about  0.1  dB  at  special  frequency  points. 

There  are  two  telecommunication  measuring  systems 
available  in  Germany.  One  is  manufactured  by  Wandel 
&  Goltermann,  the  other  by  Siemens.  Both  systems  are 
especially  designed  for  easy  operation,  remote  control, 
programmability  of  all  essential  operating  functions, 
and  possible  use  as  an  automatic  level-measurement 
system.  Some  years  ago  it  was  adequate  to  measure 
errors  of  0.5  dB  but  the  progress  of  carrier-frequency 
technology  now  requires  measurement  errors  no  greater 
than  0.1  dB  at  frequencies  up  to  12  or  18  MHz. 

The  assembly  manufactured  by  Wandel  &  Golter- 
mann is  constructed  principally  for  laboratories  and  in- 
cludes a  level  generator.  The  generator  and  the  level 
meter  are  synchronously  tuned  from  a  control  oscillator, 
which  allows  decade-frequency  settings  in  steps  of  1  Hz 
between  200  Hz  and  1,999,999  Hz.  The  absolute  ac- 
curacy, according  to  the  crystal,  is  2  X  10"8  after  one 
hour  of  service,  and  the  transient  time  for  varying  the 
frequency  is  smaller  than  100  fis.  The  generator  output 
level  can  be  selected  in  steps  of  0.1  dB  in  the  wide  range 
of  —69.9  to  9.9  dB,  indicated  as  digital  readout.  This 
output  level  has  a  measurement  error  of  0.04  dB  and,  at 
special  calibration  points,  about  0.01  dB. 
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The  same  accuracy  applies  also  to  the  level  measure- 
ment appliance.  The  manufacturer  uses  transformers 
for  these  precise  steps,  which  are  frequency-independent 
over  a  wide  range.  The  required  attenuation  is  de- 
termined exactly  by  the  turns  relationship  of  trans- 
formers and  is  not  subject  to  aging.  The  transformers 
are  connected  together  by  electronic  means. 

With  a  program  control  unit,  the  set  becomes  suitable 
for  high-precision  level  and  attenuation  measurements 
in  development  laboratories,  construction  plants,  and 
test  departments  as  well  as  for  monitoring  transmission 
systems  in  telecommunication  technology.  All  essential 
adjustments  on  the  individual  instruments  of  the  level- 
measuring  set,  and  ultimately  on  the  test  object,  can  be 
remote  controlled.  The  high  measurement  rate  ( ap- 
proximately one  measurement  point  per  second  )  results 
in  great  time  savings  during  prolonged  measurement 
runs  without  affecting  the  high  measurement  accuracy 
offered  by  the  set.  Since  it  is  only  necessary  to  ex- 
change program  tapes  in  order  to  reprogram  the  meas- 
urement system  for  other  measurement  tasks,  even 
small  production  lots  and  short  measuring  runs  can  be 
carried  out  economically. 

It  is  possible,  during  the  evaluation  process,  to  com- 
pare the  measured  test  results  with  tolerances  given  on 
the  program  tape.  The  test-point  number,  the  classifica- 
tion of  the  sample,  or  a  tolerance  violation  are  directly 
indicated  on  the  front  panel  and/or  recorded  by  tele- 
printer, electric  typewriter,  or  tape  punch.  On  the  tape, 
any  5-  or  8-unit  code  can  be  used,  with  a  maximum 
reading  speed  of  1000  characters  per  second.  The 
reading  time  for  an  average  program  will  be  approxi- 
mately 0.5  s. 

The  control  of  the  various  attenuators  and  the  power- 
level  calibration  are  made  automatically  with  the  aid  of 
the  control  unit.  The  power  level  available  at  the  out- 
put terminals,  therefore,  is  always  referred  to  the  se- 
lected output-impedance  value.  The  operation  is  really 
simplified,  since  the  calibration  occurs  automatically 
during  the  measurement  process.  The  automatic  meas- 
uring system  can  be  matched  to  a  variety  of  measure- 
ment tasks  with  the  aid  of  further  accessories.  To  illus- 
trate the  test  speed,  an  acceptance  test  of  a  group  filter 
can  be  carried  out  in  about  2  min.  The  program  can  be 
stopped  after  any  time  interval. 

The  other  manufacturer,  Siemens,  has  chosen  a  sys- 
tem with  a  great  number  of  special  appliances  which  can 
be  combined  with  a  large  number  of  measuring  sets  ac- 
cording to  the  tasks.  The  main  appliances  are  still  the 
generator,  the  level  measuring  set,  the  switching  device, 
the  attenuator,  and  all  the  different  sets  for  remote  con- 
trol and  display,  available  either  by  typing  in  any  form 
or  by  showing  on  a  screen.  The  accuracy  is  as  good  as 
the  equipment  mentioned  first. 

Special  efforts  were  made  for  the  switching  device, 
which  is  able  to  connect  the  appropriate  units  with  the 
different  devices  either  manually  or  automatically  up  to 
a  frequency  of  2  MHz,  with  coaxial  impedances  of  75 
or  100  fl  and  with  symmetrical  impedances  of  15  or 
600  H.  In  the  range  up  to  100  MHz  only  the  150-fi 
impedance  is  used.  The  system  is  constructed  using 
new  techniques  for  precision  and  for  easy  operation. 

Another  most  important  part  of  the  Siemens  assem- 


bly is  the  variable  attenuator,  of  a  really  high  accuracy, 
which  combines  a  rather  large  number  of  single  ele- 
ments. These  are  specially  designed  resistors  on  glass 
carriers,  constructed  by  a  new  technology,  which  do  not 
change  their  values  in  time,  space,  or  with  frequency. 
The  steps  used  in  the  attenuator  are  also  0.1  dB,  but 
each  element  is  calibrated  to  a  maximum  0.003-dB 
error.  You  should  note  that  this  is  not  a  calibration  in 
a  laboratory  but  is  valid  in  the  production  stage.  The 
set  can  be  used  for  many  purposes — in  the  very  early 
stage  of  newly  developed  equipment,  to  survey  the  out- 
come of  production,  to  register  the  relevant  statistical 
endurance,  and  also  in  the  field  of  research. 

In  tin-  neai  Inline  the  enlargement  of  the  national 
telecommunication  systems  will  be  aided  also  by  a 
rapidly  increasing  number  of  data-exchange  lines  with 
wider  bands  and  with  increased  transmission  require- 
ments. Last  but  not  least,  we  are  standing  at  the  begin- 
ning of  the  Picturephone  era.  This  will  bring  us  larger 
networks  with  still  wider  bandwidths. 

The  German  postal  service,  PTT,  like  that  of  other 
nations,  was  confronted  with  these  problems  and  with 
the  lack  of  staffs  trained  for  this  job.  It  was  therefore 
necessary  to  develop  an  automatic  system  and  to  meas- 
ure all  the  different  lines  from  only  a  few  measurement 
centers.  This  system  is  named  "Automatische  Mess- 
wert-Ubertragung,"  meaning  automatic  transmission 
of  measured  values.  It  operates  in  the  following  man- 
ner. A  code  number  must  be  given  to  each  line  or 
channel— not  to  the  whole  transmission  link— in  order  to 
select  this  specific  channel  by  remotely  controlled  auto- 
matic switches,  separated  from  the  normal  switches  in 
automatic  exchanges.  Remotely  controlled,  highly  pre- 
cise measurements  are  possible  in  the  field  of  noise  level, 
effective  equivalence,  frequency  response,  group  delay, 
effective  attenuation  of  termination  circuits,  and  insta- 
bility of  emplifiers  by  measuring  the  return  loss.  All 
these  measurements  are  remotely  controlled,  so  that  it 
is  not  necessary  to  have  a  person  at  the  far  end  of  the 
line.  The  remote  control  unit  and  the  line  selector  will 
not  be  discussed  here,  but  examples  of  the  measurement 
principles  will  be  mentioned. 
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Figure  1.  Measurement  of  effective  equivalence. 
C  =  control  unit;  R  =  resistor  (600— ft);  G  =  generator; 
M=branch  to  be  measured;  LM=  level  meter;  FM=  frequency 
modulation  and  demodulation;   D=decoder;  P=printer/dis- 
play;  A=automatically  controlled  amplifier. 
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As  shown  in  figure  1,  the  measurement  for  the  effec- 
tive equivalence  is  done  first  for  the  transmitting  direc- 
tion. The  generator  is  switched  to  the  near  end  and  the 
level  measurement  equipment  to  the  far  end.  The  re- 
sults are  coded  and  transmitted  by  frequency-modulated 
signals  via  the  second  (receiving)  end  of  the  line. 

The  measurement  of  the  effective  equivalence  of  the 
receiving  side  of  the  line  also  is  done  by  control  at  the 
near  end.  Instead  of  the  level  measurement  equipment, 
an  amplifier  with  automatic  gain  control  will  be 
switched  to  the  far  end  of  the  transmitting  line.  This 
amplifier  transmits  all  controlled  signals  to  the  far  end 
of  the  receiving  branch  so  that  this  side  can  be  meas- 
ured. The  level  meter  is  then  switched  to  the  far  end  of 
the  receiving  branch  so  that  this  side  can  be  measured. 
The  level  meter  is  then  switched  to  the  near  ( receiving ) 
end  of  the  line. 

For  the  noise  level  measurement  ( fig.  2 ) ,  the  trans- 
mitting near  end  is  switched  to  a  resistor  (600  ft)  and 
the  far  end  of  the  transmitting  branch  is  connected  to 
the  level  measurement  equipment.  The  values  are  trans- 
mitted, as  before,  over  the  receiving  branch  and  are 
displayed  in  the  same  way.  To  measure  the  noise  level 
on  the  receiving  branch,  the  far  end  of  this  branch  is 
switched  to  the  balance  resistor  ( 600  ft ) ,  and  the  level 
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Figure  2.  Measurement  of  noise  level. 
decontrol  unit;  R  =  resistor  (600-0*  ;  G  =  generator; 
M=branch  to  be  measured;  LM=  level  meter;  FM=  frequency 
modulation  and  demodulation;   D=decoder;  P=printer/dis- 
play;  A=automatically  controlled  amplifier. 


is  measured  at  the  near  end.  The  transmitting  branch 
is  used  only  for  remote  control. 

The  high  precision  necessary  for  reliable  measure- 
ments makes  it  necessary  to  incorporate  in  the  program 
a  surveillance  of  the  automatic  controlled  amplifier. 
This  is  done  (fig.  3)  in  the  following  way.  The  gener- 
ator at  the  near  end  works  to  the  amplifier  at  the  far 
end.  The  far-end  level  measuring  equipment  is  con- 
nected directly  to  the  output  of  the  amplifier  and  trans- 
mits the  results  in  the  usual  manner  to  the  control  unit. 
This  check  of  the  accuracy  of  the  controlled  amplifiers 
becomes  even  more  important  when  a  line  to  be  meas- 
ured does  not  end  at  the  measuring  center  but  must  be 
switched  to  the  measuring  center  through  one  or  two 
other  lines. 
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Figure  3.  Calibration  control 

C=control  unit;  R  =  resistor  (600-fi);  G  =  generator; 
M=branch  to  be  measured;  LM=  level  meter;  FM=  frequency 
modulation  and  demodulation;  D=decoder;  P=printer/dis- 
play;  A=automatically  controlled  amplifier. 


All  this  equipment  could  be  constructed  only  accord- 
ing to  the  new  electronic  technologies,  which  permit 
equipment  to  be  small,  light,  reliable,  and  precise.  The 
special  design  of  transformer  or  resistor  attenuators, 
together  with  automatic  calibration  and  comparison 
allows  remote  control  of  the  whole  measurement  proce- 
dure in  production  tests  as  well  as  in  acceptance  tests. 
Hours  of  highly  qualified  staff  members  are  saved  for 
further  development  work.  Hours  of  production  are 
saved  to  supply  the  telecommunication  companies  with 
the  necessary  appliances  and  components  for  the  exten- 
sion of  networks  to  serve  the  progress  of  economics. 

Automatic  maintenance  enables  the  companies  to  in- 
stall wideband  transmission  lines  necessary  for  data 
exchange  with  the  necessary  reliability  for  this  special 
purpose.  Economical  and  technical  progress  are  de- 
pendent, to  a  very  great  extent,  on  good  communications 
and  better  data  exchange. 
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The  national  electrical  standards  for  Japan  described  herein  are  established  and  maintained  in  the 
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1.  Introduction 

The  requirement  for  better  standards  and  their 
traeceability  to  the  national  standards  has  been  em- 
phasized in  Japan.  This  is  most  significant  in  elec- 
trical standards.  The  national  electrical  standards 
(and  those  for  photometry,  acoustics,  and  ionizing  ra- 
diation) were  established  and  are  maintained  in  the 
Electrotechnical  Laboratory  ( ETL ) ,  in  this  country. 
Research  on  the  standards  has  been  continued  in  ETL. 

The  electrical  standards  are  supplied  throughout 
Japan  by  two  major  public  agencies  and  their  branches. 
Many  activities  have  been  continuously  supporting  the 
flow  of  the  standards.  Committees  on  traceability  and 
measuring  techniques  have  been  actively  operating  in 
the  Institute  of  Electrical  Engineers  of  Japan.  A  gov- 
ernment committee  has  been  established  to  improve  the 
connection  between  national  standards  laboratories  and 
industry.  Improvement  in  international  contacts  re- 
garding this  problem  is  desired,  and  establishment  of  an 
organization  similar  to  the  NCSL  is  being  considered. 

2.  Electrical  Standards  in  the  ETL 
2.1.  Basic  Electrical  Standards 

(1)  Resistance  and  voltage:  Standard  values  of  re- 
sistance and  voltage  are  maintained  by  10  resistors  and 
21  cells.  They  have  been  checked  by  regular  interna- 
tional comparisons.  Measuring  accuracy  of  0.01  ppm  has 
been  obtained.  From  experience  gained  with  extensive 
testing  of  many  standard  cells,  criteria  for  selection  of 
standard  cells  for  the  bank  of  21  cells  were  established. 

(2)  Capacitance:  A  standard  value  has  been  de- 
termined absolutely  by  a  cross  capacitor  with  an  un- 
certainty of  1  ppm.  An  improved  cross  capacitor  is 
now  completed.  Good  results  were  obtained  in  interna- 
tional comparisons  of  0.1-/jF  and  10-pF  capacitors. 
The  excellent  stability  of  the  0.1-juF  capacitor  at  ETL 
was  recognized.  Fused-quartz  capacitors,  and  improved 
fused-quartz  tubular  capacitors  have  also  been  used. 

(3)  Ac—dc  comparison:  Electrostatic  and  thermal 
comparators  have  been  tested.    Differences  of  ac—dc 


response  have  been  measured  with  accuracy  better  than 
25  ppm  at  4  kHz,  and  40  ppm  at  20  kHz.  Extension  of 
frequency  range  is  being  tried.  Preparation  for  the 
planned  international  comparison  has  almost  been 
completed. 

(4)  Automatic  measuring  system  for  standard  cells: 
The  system  has  been  completed  to  monitor  the  standard 
cells.  It  is  composed  of  a  scanner,  digital-voltmeter  and 
a  programmer,  and  can  compare  200  cells  with  a  reso- 
lution of  0.1  fi\.  Presently,  data  are  treated  in  a  sep- 
arate computer  but  a  complete  on-line  system  is  planned. 
Extension  of  the  system  to  other  standards  is  also  under 
consideration. 

2.2.  Absolute  Measurements 

(1 )  Determination  of  capacitance  by  cross  capacitor: 
Cross  capacitor  No.  2  has  been  completed.  An  accuracy 
of  10~7  has  been  verified  by  tests.  The  capacitor  is 
shown  in  figure  1. 


Figure  1.  Improved  horizontal  cross  capacitor  (No.  2). 
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l2l  Evaluation  of  resistance  by  capacitance  and  fre- 
quency: Based  on  the  capacitance  standard,  such  meas- 
urements are  being  prepared,  using  an  improved  quad- 
rature bridge. 

I  3  I  Evaluation  of  voltage  by  an  electrostatic  balance: 
A  method  of  measuring  force  produced  by  an  incre- 
mental energy  change  in  a  movable  capacitance  elec- 
trode system  has  been  studied.  A  balance,  with  the 
electrode  system  and  a  measuring  system  for  high  volt- 
age are  being  constructed. 

2.3.  Quantum  Electric  Standards 

( 1 1  Measurement  of  p':  The  value  of  p'  (gyromag- 
netic  coefficient  of  the  proton  )  has  been  determined  with 
an  accuracy  of  ±4  ppm.  The  free  precessional  fre- 
quency of  a  proton  in  water,  in  a  calculable  magnetic 
field  produced  by  a  single  layer  air-core  solenoid,  is 
measured.  A  new  system  for  a  quantum  standard  of 
current  has  been  planned  this  year.  In  this  method  a 
new  device  called  the  Magnetically  Isolated  Calculable 
Solenoid  (MICS)  will  be  used.  The  MICS  has  a  very 
wide  constant  and  calculable  magnetic  field  which  is 
not  affected  by  ambient  fiejd  variations.  If  this 
method  goes  well,  a  quantum  standard  of  current  with- 


out the  need  for  a  non-magnetic  environment  can  be 
expected. 

(2)  Stark  voltmeter:  A  quantum  standard  of  voltage 
by  determining  the  Stark  separation  coefficient  of  parti- 
cles has  been  studied.  An  accuracy  of  ±30  ppm  was 
obtained  in  1969,  by  measuring  the  spectrum  (20  GHz ) 
of  s-trioxane  (C3  Hr,  03).  The  spectrum  of  methyl  cy- 
anide (CH3  CN  10)  at  36  GHz  is  also  being  investigated 
in  an  effort  to  obtain  higher  accuracy. 

(3)  Ac  Josephson  voltage  standard:  Research  on  a 
pure  quantum  standard  of  voltage,  using  the  Josephson 
effect  in  a  junction  of  two  superconductors,  was  started 
in  1969.  It  now  is  in  the  primary  stage  of  study  on  how 
to  make  the  tunnel  junction. 

2.4.  Radiofrequency  Standards 

Research  and  development  concerning  rf  measure- 
ments and  standards  was  started  at  ETL  in  1952.  Since 
then  a  great  deal  of  effort  has  been  expended  towards 
basic  research  and  development  of  the  national  stand- 
ards for.  and  the  measurements  of,  various  fundamental 
quantities  in  the  radio  spectrum.  These  high-frequency, 
microwave,  and  millimeter-wave  standards  and  meas- 
urements include  such  items  as  power,  voltage,  current, 
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noise,  attenuation,  phase  shift,  and  impedance.  Their 
present  status  is  shown  in  table  1. 

Since  1959,  ETL  has  made  many  efforts  towards  the 
achievement  of  agreement  among  the  standards  of  dif- 
ferent nations.  We  have  actively  participated  in  the 
intercomparisons  of  such  standards  as  10-GHz  wave- 
guide power,  400-MHz  coaxial  power,  35-GHz  milli- 
meter-wave power,  10-GHz  noise  temperature,  3-GHz 
coaxial  power,  and  rf  attenuation  at  three  different  fre- 
quencies. 

3.  Traceability  of  Electrical  Standards 
in  Japan 

In  Japan  the  traceability  route  is  split  in  two,  mainly 
for  historical  reasons.  The  dc  and  low-frequency 
standards  are  distributed  by  the  Japan  Electric  Meters 
Inspection  Corporation  (JEMIC),  and  the  radiofre- 
quency  standards  by  the  Japan  Machinery  and  Metals 
Inspection  Institute  (JMI),  as  shown  in  figure  2,  where 
it  is  compared  to  that  of  USA. 

The  standards  are  compared  through  their  rural 
branches  to  standards  of  other  organizations  as  main- 
tained in  their  own  central  standards  laboratories. 
Measuring  instruments  of  the  organizations  also  are  cal- 
ibrated at  their  standards  laboratories.  However,  many 
instruments  are  brought  to  the  corporations  directly 
from  smaller  companies. 
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Figure  2.  Traceability  of  electrical  standards  in 
Japan  and  the  United  States 

3.1  Japan  Electric  Meters  Inspection 
Corporation  (JEMIC) 

The  JEMIC  is  a  public  corporation  established  in 
1965.  The  activities  of  the  JEMIC  are  regulated  by  law. 
That  is  to  say,  the  function  of  the  Corporation  consists 
of  three  main  activities,  namely  ( 1 )  type-approval  test 
and  individual  inspection  of  watt-hour  meters,  ( 2  )  cal- 
ibration services  for  electrical  standards  and  measuring 
instruments,  (3)  research  and  investigations  for  elec- 
trical measurements. 

The  calibration  services  for  electrical  standards  and 
measuring  instruments  were  transferred  from  ETL  to 
JEMIC.  At  present,  ETL  does  not  perform  direct  cali- 
bration services  at  dc  and  in  the  low-frequency  region, 
but  JEMIC  supplies  the  standards  as  maintained  by 


ETL  to  all  the  industries  and  the  government  offices  in 
Japan.  Moreover,  ETL  maintains  only  basic  standards 
such  as  standard  resistors,  standard  cells,  standard 
capacitors,  and  ac-dc  transfer  standards.  So,  the 
JEMIC  develops  some  practical  standards  independ- 
ently, such  as  standard  inductors,  ratio  sets,  shunts, 
voltboxes,  instrument  transformers,  inductive  voltage 
dividers,  standard  watt-hour  meters,  etc. 

At  present,  JEMIC  has  1300  personnel.  Of  these, 
about  120  are  employed  on  standards  work  both  for  its 
own  use  and  calibration  services.  JEMIC  covers  the 
whole  service  area  in  Japan  through  fifteen  branch 
laboratories  located  all  over  the  country.  The  JEMIC 
calibrated  8700  articles  by  request  in  1969. 

3.2.  Japan  Machinery  and  Metals 
Inspection  Institute  (JMI) 

The  JMI  is  a  nonprofit  public  agency  founded  in 
1957,  subsequent  to  the  Export  Inspection  Law.  The 
Department  of  Calibration  of  Electronic  Measuring  In- 
struments was  established  in  1963,  as  one  of  the  agencies 
to  transfer  the  national  standards  of  ETL  to  official  and 
private  organization  users. 

The  main  business  of  JMI  is  ( 1 )  calibration  of 
standards  and  measuring  instruments  in  the  radiofre- 
quency  ranges,  and  their  adjustment  and  repair,  (2) 
general  inspection  of  machinery,  metals,  electrical  and 
electronic  products,  particularly  inspection  of  equip- 
ment for  export,  (3)  safety  tests  by  CSA  and  UL,  and 
spurious-radiation  tests  by  FCC  rules. 

The  JMI  has  550  persons  and  a  branch  in  Osaka.  Re- 
search for  better  calibration  has  been  continued.  Main 
instruments  for  calibration  are  signal  generators,  hf 
ammeters,  power  and  impedance  meters,  Q  meters,  dis- 
tortion-factor meters,  wow-flutter  meters,  electrical 
standards  and  indicating  meters. 

4.  Committees 
4.1  Traceability  Committee 

In  March  1968.  the  Institute  of  Electrical  Engineers 
of  Japan  set  up  the  Traceability  Committee  to  investi- 
gate, to  discuss,  and  to  solve  the  problem  of  traceability. 
The  members  of  this  committee  are  the  metrology  sci- 
entists and  engineers  of  ETL.  of  public  standards  labo- 
ratories, and  of  the  users  and  makers  of  electrical  and 
electronic  measuring  instruments.  The  major  activities 
of  this  committee  are  as  follows: 

( 1 )  Disseminating  the  research  data  concerning  legal 
standards  and  international  comparisons. 

(2)  Informing  committee  members  about  the  work 
and  the  standards-supply  systems  of  the  national  and 
public  standards  laboratories, 

(3)  Collecting  and  disseminating  information  about 
standards  systems  in  other  countries, 

(4)  Investigation  of  standards-maintenance  systems 
of  the  industries, 

(5)  Investigation  of  the  conditions  of  standards- 
laboratory  rooms, 

(6)  Standardization  of  the  calibration  methods  of 
electrical  and  electronic  standards  and  instruments, 

(7)  Presentation  of  new  instruments. 
This  committee  meets  regularly  every  month. 
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4.2.  Precision  Electromagnetic 
Measurements  Committee 

This  committee  was  established  in  April  1967,  in  the 
Institute  of  Electrical  Engineers  of  Japan,  to  exchange 
novel  techniques  for  precision  measurements  among 
people  of  ETL,  agencies  of  inspection,  and  professors 
and  engineers  from  the  measuring-instrument  industry. 

Topics  concerned  with  research  and  development  ba- 
sic to  applications,  and  covering  the  range  from  dc  to 
laser  frequencies,  are  presented  and  discussed.  The 
committee  meets  monthly  as  a  rule. 

4.3.  Governmental  Committee  on 
Standards  and  Measurements 

This  committee  was  established  in  February  of  1969. 
Members  include  people  from  the  Ministry  of  Interna- 
tional Trade  and  Industry,  standards  laboratories,  in- 
spection agencies,  institutes,  universities,  manufactur- 
ing and  other  industries,  and  measuring-instrument 
makers.  Seven  branch  committees  for  electricity,  il- 
lumination, units,  temperature,  ionizing  radiation, 
length,  and  standard  materials  have  been  established. 
Activities  and  objectives  of  the  committee  are: 

1 1 )  Presentation  of  information  from  national  and 


international  conferences  and  committees  on  standards, 

(2)  Exchange  of  information  on  national  standards 
laboratories  and  inspection  agencies,  and  discussion  of 
means  to  improve  their  operation, 

(3)  Collection  of  requests  from  industry  to  the  labo- 
ratories and  the  agencies, 

(4)  Discussion  of  future  general  traceability  systems. 
The  necessity  for  securing  good  traceability  and  for 

improving  measuring  techniques  in  the  industry  have 
been  pointed  out.  For  this  purpose,  establishment  of  a 
new  organization  similar  to  the  NCSL  in  the  USA  is 
being  considered. 

5.  Conclusion 

The  importance  of  favorable  supply  and  effective  ap- 
plication of  standards  have  been  recognized  in  the  in- 
dustry of  Japan.  Some  activities  have  been  established 
and  operated  for  this  purpose.  However,  such  a  trend 
is  seen  to  appear  in  many  countries.  They  may  have 
common  problems  to  be  solved.  International  contact 
is  very  necessary  and  effective  even  if  domestic  condi- 
tions are  different.  Methods  of  traceability  as  well  as 
the  standards  themselves  must  be  international  to  realize 
their  full  value. 
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Latest  developments  in  the  subject  area  of  this  publication,  as  well  as 
in  other  areas  where  the  National  Bureau  of  Standards  is  active,  are 
reported  in  the  NBS  Technical  News  Bulletin.  See  following  page. 
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